Loading KG indexer from data/tkgl-smallpedia_edgelist.csv
[Indexer] Loaded 550,376 triples from data/tkgl-smallpedia_edgelist.csv
Loading policy from ../checkpoints/sft/final

Loading weights:   0%|          | 0/399 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 399/399 [00:00<00:00, 4809.22it/s]
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
[diag] policy class: PeftModelForCausalLM
[diag] accelerate.is_peft_model: True  (must be True or TRL will load a 2nd ref model and OOM)
[diag] policy loaded; CUDA allocated = 0.00 GB (expected ~5.5 GB for QLoRA, ~16 GB for bf16)
[diag] # of 4-bit linear modules in policy: 0
  6096 training prompts
Continuing training of SFT adapter already attached to policy (not stacking a new one)
Starting T-GRPO training

  0%|          | 0/10000 [00:00<?, ?it/s]
  0%|          | 1/10000 [00:05<13:59:42,  5.04s/it]
  0%|          | 2/10000 [00:09<12:30:54,  4.51s/it]
  0%|          | 3/10000 [00:13<12:21:35,  4.45s/it]
  0%|          | 4/10000 [00:18<12:53:07,  4.64s/it]
  0%|          | 5/10000 [00:22<12:34:10,  4.53s/it]
  0%|          | 6/10000 [00:27<12:27:31,  4.49s/it]
  0%|          | 7/10000 [00:31<12:23:13,  4.46s/it]
  0%|          | 8/10000 [00:35<12:14:39,  4.41s/it]
  0%|          | 9/10000 [00:39<11:47:04,  4.25s/it]
  0%|          | 10/10000 [00:44<11:59:23,  4.32s/it]
  0%|          | 11/10000 [00:49<12:22:01,  4.46s/it]
  0%|          | 12/10000 [00:53<12:07:13,  4.37s/it]
  0%|          | 13/10000 [00:57<11:54:57,  4.30s/it]
  0%|          | 14/10000 [01:01<11:47:24,  4.25s/it]
  0%|          | 15/10000 [01:05<11:32:54,  4.16s/it]
  0%|          | 16/10000 [01:09<11:25:27,  4.12s/it]
  0%|          | 17/10000 [01:13<11:12:02,  4.04s/it]
  0%|          | 18/10000 [01:21<14:56:47,  5.39s/it]
  0%|          | 19/10000 [01:26<13:53:25,  5.01s/it]
  0%|          | 20/10000 [01:30<13:20:36,  4.81s/it]
                                                     

  0%|          | 20/10000 [01:30<13:20:36,  4.81s/it]
  0%|          | 21/10000 [01:34<12:46:38,  4.61s/it]
  0%|          | 22/10000 [01:38<12:25:25,  4.48s/it]
  0%|          | 23/10000 [01:43<12:26:55,  4.49s/it]
  0%|          | 24/10000 [01:47<12:27:03,  4.49s/it]
  0%|          | 25/10000 [01:52<12:45:03,  4.60s/it]
  0%|          | 26/10000 [01:56<12:14:53,  4.42s/it]
  0%|          | 27/10000 [02:00<12:15:41,  4.43s/it]
  0%|          | 28/10000 [02:05<12:36:39,  4.55s/it]
  0%|          | 29/10000 [02:10<12:39:20,  4.57s/it]
  0%|          | 30/10000 [02:14<12:10:58,  4.40s/it]
  0%|          | 31/10000 [02:18<12:14:27,  4.42s/it]
  0%|          | 32/10000 [02:23<12:22:12,  4.47s/it]
  0%|          | 33/10000 [02:27<12:06:12,  4.37s/it]
  0%|          | 34/10000 [02:31<11:55:04,  4.31s/it]
  0%|          | 35/10000 [02:36<12:25:39,  4.49s/it]
  0%|          | 36/10000 [02:41<12:22:43,  4.47s/it]
  0%|          | 37/10000 [02:45<12:10:04,  4.40s/it]
  0%|          | 38/10000 [02:49<12:18:16,  4.45s/it]
  0%|          | 39/10000 [02:54<12:27:28,  4.50s/it]
  0%|          | 40/10000 [02:58<11:56:49,  4.32s/it]
                                                     

  0%|          | 40/10000 [02:58<11:56:49,  4.32s/it]
  0%|          | 41/10000 [03:02<12:06:49,  4.38s/it]
  0%|          | 42/10000 [03:07<12:09:07,  4.39s/it]
  0%|          | 43/10000 [03:11<11:50:46,  4.28s/it]
  0%|          | 44/10000 [03:15<11:52:02,  4.29s/it]
  0%|          | 45/10000 [03:19<11:44:34,  4.25s/it]
  0%|          | 46/10000 [03:24<11:40:03,  4.22s/it]
  0%|          | 47/10000 [03:28<11:41:52,  4.23s/it]
  0%|          | 48/10000 [03:32<11:41:30,  4.23s/it]
  0%|          | 49/10000 [03:36<11:45:10,  4.25s/it]
  0%|          | 50/10000 [03:40<11:34:08,  4.19s/it]
  1%|          | 51/10000 [03:45<11:32:56,  4.18s/it]
  1%|          | 52/10000 [03:49<11:39:05,  4.22s/it]
  1%|          | 53/10000 [03:53<11:35:22,  4.19s/it]
  1%|          | 54/10000 [03:58<12:03:57,  4.37s/it]
  1%|          | 55/10000 [04:02<12:14:01,  4.43s/it]
  1%|          | 56/10000 [04:07<12:13:40,  4.43s/it]
  1%|          | 57/10000 [04:11<11:56:37,  4.32s/it]
  1%|          | 58/10000 [04:15<11:52:15,  4.30s/it]
  1%|          | 59/10000 [04:19<11:40:13,  4.23s/it]
  1%|          | 60/10000 [04:23<11:48:06,  4.27s/it]
                                                     

  1%|          | 60/10000 [04:23<11:48:06,  4.27s/it]
  1%|          | 61/10000 [04:28<11:39:11,  4.22s/it]
  1%|          | 62/10000 [04:32<11:46:09,  4.26s/it]
  1%|          | 63/10000 [04:36<11:41:14,  4.23s/it]
  1%|          | 64/10000 [04:40<11:41:29,  4.24s/it]
  1%|          | 65/10000 [04:44<11:34:04,  4.19s/it]
  1%|          | 66/10000 [04:49<11:38:24,  4.22s/it]
  1%|          | 67/10000 [04:53<11:35:42,  4.20s/it]
  1%|          | 68/10000 [04:57<11:50:29,  4.29s/it]
  1%|          | 69/10000 [05:02<11:55:55,  4.33s/it]
  1%|          | 70/10000 [05:06<12:09:19,  4.41s/it]
  1%|          | 71/10000 [05:11<11:55:47,  4.33s/it]
  1%|          | 72/10000 [05:15<12:01:03,  4.36s/it]
  1%|          | 73/10000 [05:19<12:04:46,  4.38s/it]
  1%|          | 74/10000 [05:24<12:19:56,  4.47s/it]
  1%|          | 75/10000 [05:28<11:59:07,  4.35s/it]
  1%|          | 76/10000 [05:32<11:49:04,  4.29s/it]
  1%|          | 77/10000 [05:37<12:00:44,  4.36s/it]
  1%|          | 78/10000 [05:41<11:45:06,  4.26s/it]
  1%|          | 79/10000 [05:45<11:27:28,  4.16s/it]
  1%|          | 80/10000 [05:49<11:38:39,  4.23s/it]
                                                     

  1%|          | 80/10000 [05:49<11:38:39,  4.23s/it]
  1%|          | 81/10000 [05:54<11:55:40,  4.33s/it]
  1%|          | 82/10000 [05:58<11:33:51,  4.20s/it]
  1%|          | 83/10000 [06:01<11:12:40,  4.07s/it]
  1%|          | 84/10000 [06:06<11:46:54,  4.28s/it]
  1%|          | 85/10000 [06:11<11:58:11,  4.35s/it]
  1%|          | 86/10000 [06:15<11:41:21,  4.24s/it]
  1%|          | 87/10000 [06:19<11:54:31,  4.32s/it]
  1%|          | 88/10000 [06:23<11:52:02,  4.31s/it]
  1%|          | 89/10000 [06:27<11:24:50,  4.15s/it]
  1%|          | 90/10000 [06:32<11:38:58,  4.23s/it]
  1%|          | 91/10000 [06:36<11:56:46,  4.34s/it]
  1%|          | 92/10000 [06:40<11:45:32,  4.27s/it]
  1%|          | 93/10000 [06:45<11:43:33,  4.26s/it]
  1%|          | 94/10000 [06:48<11:18:41,  4.11s/it]
  1%|          | 95/10000 [06:53<11:46:41,  4.28s/it]
  1%|          | 96/10000 [06:57<11:44:18,  4.27s/it]
  1%|          | 97/10000 [07:01<11:19:15,  4.12s/it]
  1%|          | 98/10000 [07:06<11:43:31,  4.26s/it]
  1%|          | 99/10000 [07:10<11:54:45,  4.33s/it]
  1%|          | 100/10000 [07:14<11:52:50,  4.32s/it]
                                                      

  1%|          | 100/10000 [07:14<11:52:50,  4.32s/it]
  1%|          | 101/10000 [07:19<11:44:23,  4.27s/it]
  1%|          | 102/10000 [07:23<11:58:09,  4.35s/it]
  1%|          | 103/10000 [07:27<11:49:52,  4.30s/it]
  1%|          | 104/10000 [07:31<11:38:28,  4.23s/it]
  1%|          | 105/10000 [07:36<11:44:46,  4.27s/it]
  1%|          | 106/10000 [07:40<11:21:15,  4.13s/it]
  1%|          | 107/10000 [07:44<11:29:47,  4.18s/it]
  1%|          | 108/10000 [07:48<11:25:40,  4.16s/it]
  1%|          | 109/10000 [07:52<11:43:21,  4.27s/it]
  1%|          | 110/10000 [07:57<11:56:55,  4.35s/it]
  1%|          | 111/10000 [08:01<11:47:53,  4.30s/it]
  1%|          | 112/10000 [08:06<12:01:02,  4.38s/it]
  1%|          | 113/10000 [08:10<11:59:47,  4.37s/it]
  1%|          | 114/10000 [08:14<11:33:59,  4.21s/it]
  1%|          | 115/10000 [08:18<11:21:01,  4.13s/it]
  1%|          | 116/10000 [08:22<11:35:20,  4.22s/it]
  1%|          | 117/10000 [08:26<11:13:19,  4.09s/it]
  1%|          | 118/10000 [08:30<11:16:41,  4.11s/it]
  1%|          | 119/10000 [08:35<11:31:58,  4.20s/it]
  1%|          | 120/10000 [08:38<11:11:01,  4.08s/it]
                                                      

  1%|          | 120/10000 [08:38<11:11:01,  4.08s/it]
  1%|          | 121/10000 [08:42<11:07:56,  4.06s/it]
  1%|          | 122/10000 [08:47<11:24:14,  4.16s/it]
  1%|          | 123/10000 [08:51<11:32:17,  4.21s/it]
  1%|          | 124/10000 [08:55<11:38:56,  4.25s/it]
  1%|▏         | 125/10000 [09:00<11:48:24,  4.30s/it]
  1%|▏         | 126/10000 [09:05<12:03:45,  4.40s/it]
  1%|▏         | 127/10000 [09:09<12:00:23,  4.38s/it]
  1%|▏         | 128/10000 [09:13<11:45:25,  4.29s/it]
  1%|▏         | 129/10000 [09:17<11:41:19,  4.26s/it]
  1%|▏         | 130/10000 [09:22<12:09:23,  4.43s/it]
  1%|▏         | 131/10000 [09:26<12:07:47,  4.42s/it]
  1%|▏         | 132/10000 [09:31<11:57:11,  4.36s/it]
  1%|▏         | 133/10000 [09:35<12:07:14,  4.42s/it]
  1%|▏         | 134/10000 [09:40<12:09:19,  4.44s/it]
  1%|▏         | 135/10000 [09:44<11:58:10,  4.37s/it]
  1%|▏         | 136/10000 [09:48<11:53:19,  4.34s/it]
  1%|▏         | 137/10000 [09:53<12:25:29,  4.54s/it]
  1%|▏         | 138/10000 [09:57<12:08:59,  4.44s/it]
  1%|▏         | 139/10000 [10:02<11:57:26,  4.37s/it]
  1%|▏         | 140/10000 [10:06<12:21:31,  4.51s/it]
                                                      

  1%|▏         | 140/10000 [10:06<12:21:31,  4.51s/it]
  1%|▏         | 141/10000 [10:11<12:18:13,  4.49s/it]
  1%|▏         | 142/10000 [10:15<12:04:30,  4.41s/it]
  1%|▏         | 143/10000 [10:19<11:48:43,  4.31s/it]
  1%|▏         | 144/10000 [10:24<12:04:58,  4.41s/it]
  1%|▏         | 145/10000 [10:28<11:51:24,  4.33s/it]
  1%|▏         | 146/10000 [10:32<11:54:23,  4.35s/it]
  1%|▏         | 147/10000 [10:37<12:07:31,  4.43s/it]
  1%|▏         | 148/10000 [10:41<12:00:34,  4.39s/it]
  1%|▏         | 149/10000 [10:45<11:45:19,  4.30s/it]
  2%|▏         | 150/10000 [10:49<11:34:40,  4.23s/it]
  2%|▏         | 151/10000 [10:54<11:47:48,  4.31s/it]
  2%|▏         | 152/10000 [10:58<11:56:38,  4.37s/it]
  2%|▏         | 153/10000 [11:02<11:41:52,  4.28s/it]
  2%|▏         | 154/10000 [11:07<12:02:36,  4.40s/it]
  2%|▏         | 155/10000 [11:12<12:03:04,  4.41s/it]
  2%|▏         | 156/10000 [11:16<12:22:16,  4.52s/it]
  2%|▏         | 157/10000 [11:21<12:11:57,  4.46s/it]
  2%|▏         | 158/10000 [11:25<12:18:57,  4.50s/it]
  2%|▏         | 159/10000 [11:30<12:33:26,  4.59s/it]
  2%|▏         | 160/10000 [11:35<12:26:20,  4.55s/it]
                                                      

  2%|▏         | 160/10000 [11:35<12:26:20,  4.55s/it]
  2%|▏         | 161/10000 [11:39<12:12:06,  4.46s/it]
  2%|▏         | 162/10000 [11:43<12:01:16,  4.40s/it]
  2%|▏         | 163/10000 [11:47<12:00:26,  4.39s/it]
  2%|▏         | 164/10000 [11:52<11:56:26,  4.37s/it]
  2%|▏         | 165/10000 [11:56<11:53:40,  4.35s/it]
  2%|▏         | 166/10000 [12:00<11:41:34,  4.28s/it]
  2%|▏         | 167/10000 [12:04<11:43:14,  4.29s/it]
  2%|▏         | 168/10000 [12:09<12:04:05,  4.42s/it]
  2%|▏         | 169/10000 [12:14<11:58:35,  4.39s/it]
  2%|▏         | 170/10000 [12:18<11:51:54,  4.35s/it]
  2%|▏         | 171/10000 [12:22<11:36:54,  4.25s/it]
  2%|▏         | 172/10000 [12:26<11:52:46,  4.35s/it]
  2%|▏         | 173/10000 [12:31<12:04:33,  4.42s/it]
  2%|▏         | 174/10000 [12:35<11:59:32,  4.39s/it]
  2%|▏         | 175/10000 [12:40<12:16:44,  4.50s/it]
  2%|▏         | 176/10000 [12:44<11:55:47,  4.37s/it]
  2%|▏         | 177/10000 [12:48<11:44:00,  4.30s/it]
  2%|▏         | 178/10000 [12:53<11:47:34,  4.32s/it]
  2%|▏         | 179/10000 [12:57<11:48:09,  4.33s/it]
  2%|▏         | 180/10000 [13:01<11:47:02,  4.32s/it]
                                                      

  2%|▏         | 180/10000 [13:01<11:47:02,  4.32s/it]
  2%|▏         | 181/10000 [13:05<11:35:50,  4.25s/it]
  2%|▏         | 182/10000 [13:10<11:40:25,  4.28s/it]
  2%|▏         | 183/10000 [13:14<11:35:17,  4.25s/it]
  2%|▏         | 184/10000 [13:18<11:36:54,  4.26s/it]
  2%|▏         | 185/10000 [13:22<11:19:22,  4.15s/it]
  2%|▏         | 186/10000 [13:27<11:36:00,  4.26s/it]
  2%|▏         | 187/10000 [13:31<11:27:38,  4.20s/it]
  2%|▏         | 188/10000 [13:35<11:26:56,  4.20s/it]
  2%|▏         | 189/10000 [13:40<11:54:09,  4.37s/it]
  2%|▏         | 190/10000 [13:44<11:45:55,  4.32s/it]
  2%|▏         | 191/10000 [13:48<11:32:59,  4.24s/it]
  2%|▏         | 192/10000 [13:52<11:36:13,  4.26s/it]
  2%|▏         | 193/10000 [13:57<11:45:40,  4.32s/it]
  2%|▏         | 194/10000 [14:00<11:24:13,  4.19s/it]
  2%|▏         | 195/10000 [14:05<11:23:55,  4.19s/it]
  2%|▏         | 196/10000 [14:10<12:00:08,  4.41s/it]
  2%|▏         | 197/10000 [14:14<11:48:48,  4.34s/it]
  2%|▏         | 198/10000 [14:18<11:27:55,  4.21s/it]
  2%|▏         | 199/10000 [14:22<11:39:06,  4.28s/it]
  2%|▏         | 200/10000 [14:27<12:01:02,  4.41s/it]
                                                      
{'loss': '0.1128', 'grad_norm': '3.817', 'learning_rate': '4.991e-06', 'num_tokens': '5713', 'completions/mean_length': '5.106', 'completions/min_length': '3.8', 'completions/max_length': '8.65', 'completions/clipped_ratio': '0.00625', 'completions/mean_terminated_length': '4.743', 'completions/min_terminated_length': '3.8', 'completions/max_terminated_length': '6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4325', 'rewards/reward_semantic/std': '0.1475', 'rewards/reward_length/mean': '-0.05505', 'rewards/reward_length/std': '0.01862', 'reward': '1.127', 'reward_std': '0.04492', 'frac_reward_zero_std': '0.3', 'kl': '0.002807', 'entropy': '0.7961', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.495', 'epoch': '0.003281'}
{'loss': '0.05919', 'grad_norm': '0.001022', 'learning_rate': '4.981e-06', 'num_tokens': '1.174e+04', 'completions/mean_length': '5.044', 'completions/min_length': '3.65', 'completions/max_length': '7.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.044', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '7.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2015', 'rewards/reward_semantic/std': '0.06014', 'rewards/reward_length/mean': '-0.06421', 'rewards/reward_length/std': '0.01753', 'reward': '1.057', 'reward_std': '0.01862', 'frac_reward_zero_std': '0.2', 'kl': '0.006444', 'entropy': '0.9064', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.384', 'epoch': '0.006562'}
{'loss': '0.01734', 'grad_norm': '4.661', 'learning_rate': '4.971e-06', 'num_tokens': '1.755e+04', 'completions/mean_length': '4.494', 'completions/min_length': '3.65', 'completions/max_length': '5.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.494', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '5.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4559', 'rewards/reward_semantic/std': '0.1309', 'rewards/reward_length/mean': '-0.05459', 'rewards/reward_length/std': '0.01653', 'reward': '1.134', 'reward_std': '0.0395', 'frac_reward_zero_std': '0.25', 'kl': '0.02934', 'entropy': '0.6213', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.257', 'epoch': '0.009843'}
{'loss': '0.04084', 'grad_norm': '4.343', 'learning_rate': '4.961e-06', 'num_tokens': '2.369e+04', 'completions/mean_length': '4.588', 'completions/min_length': '3.55', 'completions/max_length': '6.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.588', 'completions/min_terminated_length': '3.55', 'completions/max_terminated_length': '6.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3124', 'rewards/reward_semantic/std': '0.1395', 'rewards/reward_length/mean': '-0.0583', 'rewards/reward_length/std': '0.01756', 'reward': '1.091', 'reward_std': '0.0423', 'frac_reward_zero_std': '0.2', 'kl': '0.07788', 'entropy': '0.7305', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.263', 'epoch': '0.01312'}
{'loss': '0.0438', 'grad_norm': '2.388', 'learning_rate': '4.951e-06', 'num_tokens': '2.973e+04', 'completions/mean_length': '5.169', 'completions/min_length': '3.6', 'completions/max_length': '6.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.169', 'completions/min_terminated_length': '3.6', 'completions/max_terminated_length': '6.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3705', 'rewards/reward_semantic/std': '0.1176', 'rewards/reward_length/mean': '-0.06189', 'rewards/reward_length/std': '0.01646', 'reward': '1.108', 'reward_std': '0.03531', 'frac_reward_zero_std': '0.2', 'kl': '0.01863', 'entropy': '0.7101', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.245', 'epoch': '0.0164'}
{'loss': '0.06543', 'grad_norm': '0.009295', 'learning_rate': '4.941e-06', 'num_tokens': '3.522e+04', 'completions/mean_length': '4.056', 'completions/min_length': '3.15', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.056', 'completions/min_terminated_length': '3.15', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4037', 'rewards/reward_semantic/std': '0.1241', 'rewards/reward_length/mean': '-0.04651', 'rewards/reward_length/std': '0.008918', 'reward': '1.119', 'reward_std': '0.03756', 'frac_reward_zero_std': '0.3', 'kl': '0.03001', 'entropy': '0.6822', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.183', 'epoch': '0.01969'}
{'loss': '0.014', 'grad_norm': '3.717', 'learning_rate': '4.931e-06', 'num_tokens': '4.11e+04', 'completions/mean_length': '4.906', 'completions/min_length': '3.6', 'completions/max_length': '6.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.906', 'completions/min_terminated_length': '3.6', 'completions/max_terminated_length': '6.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3675', 'rewards/reward_semantic/std': '0.123', 'rewards/reward_length/mean': '-0.05803', 'rewards/reward_length/std': '0.01343', 'reward': '1.107', 'reward_std': '0.03725', 'frac_reward_zero_std': '0.15', 'kl': '0.0605', 'entropy': '0.7443', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.378', 'epoch': '0.02297'}
{'loss': '0.01656', 'grad_norm': '6.069', 'learning_rate': '4.921e-06', 'num_tokens': '4.734e+04', 'completions/mean_length': '4.931', 'completions/min_length': '3.55', 'completions/max_length': '7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.931', 'completions/min_terminated_length': '3.55', 'completions/max_terminated_length': '7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3149', 'rewards/reward_semantic/std': '0.2088', 'rewards/reward_length/mean': '-0.06606', 'rewards/reward_length/std': '0.01742', 'reward': '1.091', 'reward_std': '0.0627', 'frac_reward_zero_std': '0.15', 'kl': '0.05209', 'entropy': '0.633', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.388', 'epoch': '0.02625'}
{'loss': '0.1097', 'grad_norm': '0.02703', 'learning_rate': '4.911e-06', 'num_tokens': '5.316e+04', 'completions/mean_length': '4.5', 'completions/min_length': '3.65', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.5', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2206', 'rewards/reward_semantic/std': '0.09846', 'rewards/reward_length/mean': '-0.05598', 'rewards/reward_length/std': '0.01286', 'reward': '1.063', 'reward_std': '0.03002', 'frac_reward_zero_std': '0.25', 'kl': '0.08403', 'entropy': '0.7635', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.317', 'epoch': '0.02953'}

  2%|▏         | 200/10000 [14:27<12:01:02,  4.41s/it]
  2%|▏         | 201/10000 [14:31<11:48:34,  4.34s/it]
  2%|▏         | 202/10000 [14:35<11:45:28,  4.32s/it]
  2%|▏         | 203/10000 [14:40<11:51:01,  4.35s/it]
  2%|▏         | 204/10000 [14:44<11:51:07,  4.36s/it]
  2%|▏         | 205/10000 [14:48<11:26:21,  4.20s/it]
  2%|▏         | 206/10000 [14:52<11:24:50,  4.20s/it]
  2%|▏         | 207/10000 [14:57<11:38:00,  4.28s/it]
  2%|▏         | 208/10000 [15:01<11:26:51,  4.21s/it]
  2%|▏         | 209/10000 [15:05<11:25:43,  4.20s/it]
  2%|▏         | 210/10000 [15:09<11:42:56,  4.31s/it]
  2%|▏         | 211/10000 [15:14<11:43:23,  4.31s/it]
  2%|▏         | 212/10000 [15:18<11:44:05,  4.32s/it]
  2%|▏         | 213/10000 [15:23<12:07:44,  4.46s/it]
  2%|▏         | 214/10000 [15:27<11:54:32,  4.38s/it]
  2%|▏         | 215/10000 [15:31<11:42:16,  4.31s/it]
  2%|▏         | 216/10000 [15:36<11:59:57,  4.42s/it]
  2%|▏         | 217/10000 [15:41<12:23:54,  4.56s/it]
  2%|▏         | 218/10000 [15:45<12:21:39,  4.55s/it]
  2%|▏         | 219/10000 [15:50<12:20:37,  4.54s/it]
  2%|▏         | 220/10000 [15:55<12:46:51,  4.70s/it]
                                                      

  2%|▏         | 220/10000 [15:55<12:46:51,  4.70s/it]
  2%|▏         | 221/10000 [15:59<12:24:38,  4.57s/it]
  2%|▏         | 222/10000 [16:03<12:04:38,  4.45s/it]
  2%|▏         | 223/10000 [16:07<11:38:34,  4.29s/it]
  2%|▏         | 224/10000 [16:13<12:38:04,  4.65s/it]
  2%|▏         | 225/10000 [16:17<12:16:27,  4.52s/it]
  2%|▏         | 226/10000 [16:21<12:00:21,  4.42s/it]
  2%|▏         | 227/10000 [16:26<12:08:53,  4.47s/it]
  2%|▏         | 228/10000 [16:30<11:38:53,  4.29s/it]
  2%|▏         | 229/10000 [16:34<11:34:08,  4.26s/it]
  2%|▏         | 230/10000 [16:38<11:33:50,  4.26s/it]
  2%|▏         | 231/10000 [16:42<11:30:58,  4.24s/it]
  2%|▏         | 232/10000 [16:47<11:46:30,  4.34s/it]
  2%|▏         | 233/10000 [16:51<11:32:58,  4.26s/it]
  2%|▏         | 234/10000 [16:55<11:32:06,  4.25s/it]
  2%|▏         | 235/10000 [16:59<11:34:31,  4.27s/it]
  2%|▏         | 236/10000 [17:04<11:38:57,  4.30s/it]
  2%|▏         | 237/10000 [17:08<11:34:37,  4.27s/it]
  2%|▏         | 238/10000 [17:12<11:46:09,  4.34s/it]
  2%|▏         | 239/10000 [17:17<11:48:48,  4.36s/it]
  2%|▏         | 240/10000 [17:21<11:36:12,  4.28s/it]
                                                      

  2%|▏         | 240/10000 [17:21<11:36:12,  4.28s/it]
  2%|▏         | 241/10000 [17:25<11:37:20,  4.29s/it]
  2%|▏         | 242/10000 [17:29<11:29:02,  4.24s/it]
  2%|▏         | 243/10000 [17:34<11:41:17,  4.31s/it]
  2%|▏         | 244/10000 [17:38<11:33:20,  4.26s/it]
  2%|▏         | 245/10000 [17:47<15:03:53,  5.56s/it]
  2%|▏         | 246/10000 [17:51<14:01:41,  5.18s/it]
  2%|▏         | 247/10000 [17:55<12:58:54,  4.79s/it]
  2%|▏         | 248/10000 [17:59<12:52:44,  4.75s/it]
  2%|▏         | 249/10000 [18:04<12:39:59,  4.68s/it]
  2%|▎         | 250/10000 [18:08<12:24:09,  4.58s/it]
  3%|▎         | 251/10000 [18:12<11:55:02,  4.40s/it]
  3%|▎         | 252/10000 [18:17<12:03:22,  4.45s/it]
  3%|▎         | 253/10000 [18:21<11:52:28,  4.39s/it]
  3%|▎         | 254/10000 [18:25<11:47:46,  4.36s/it]
  3%|▎         | 255/10000 [18:30<12:09:54,  4.49s/it]
  3%|▎         | 256/10000 [18:35<12:05:36,  4.47s/it]
  3%|▎         | 257/10000 [18:39<11:38:41,  4.30s/it]
  3%|▎         | 258/10000 [18:43<11:32:12,  4.26s/it]
  3%|▎         | 259/10000 [18:48<11:59:42,  4.43s/it]
  3%|▎         | 260/10000 [18:52<11:46:30,  4.35s/it]
                                                      

  3%|▎         | 260/10000 [18:52<11:46:30,  4.35s/it]
  3%|▎         | 261/10000 [18:56<11:50:13,  4.38s/it]
  3%|▎         | 262/10000 [19:01<12:01:00,  4.44s/it]
  3%|▎         | 263/10000 [19:05<11:45:14,  4.35s/it]
  3%|▎         | 264/10000 [19:09<11:43:47,  4.34s/it]
  3%|▎         | 265/10000 [19:13<11:36:15,  4.29s/it]
  3%|▎         | 266/10000 [19:18<11:48:56,  4.37s/it]
  3%|▎         | 267/10000 [19:22<11:39:31,  4.31s/it]
  3%|▎         | 268/10000 [19:26<11:20:00,  4.19s/it]
  3%|▎         | 269/10000 [19:30<11:34:07,  4.28s/it]
  3%|▎         | 270/10000 [19:34<11:15:48,  4.17s/it]
  3%|▎         | 271/10000 [19:39<11:24:25,  4.22s/it]
  3%|▎         | 272/10000 [19:43<11:49:00,  4.37s/it]
  3%|▎         | 273/10000 [19:48<11:55:48,  4.42s/it]
  3%|▎         | 274/10000 [19:52<11:50:56,  4.39s/it]
  3%|▎         | 275/10000 [19:56<11:39:12,  4.31s/it]
  3%|▎         | 276/10000 [20:01<11:45:59,  4.36s/it]
  3%|▎         | 277/10000 [20:05<11:39:16,  4.32s/it]
  3%|▎         | 278/10000 [20:09<11:22:04,  4.21s/it]
  3%|▎         | 279/10000 [20:13<11:20:37,  4.20s/it]
  3%|▎         | 280/10000 [20:18<11:40:28,  4.32s/it]
                                                      

  3%|▎         | 280/10000 [20:18<11:40:28,  4.32s/it]
  3%|▎         | 281/10000 [20:22<11:27:49,  4.25s/it]
  3%|▎         | 282/10000 [20:26<11:31:31,  4.27s/it]
  3%|▎         | 283/10000 [20:31<11:52:23,  4.40s/it]
  3%|▎         | 284/10000 [20:35<11:45:43,  4.36s/it]
  3%|▎         | 285/10000 [20:39<11:31:55,  4.27s/it]
  3%|▎         | 286/10000 [20:43<11:13:40,  4.16s/it]
  3%|▎         | 287/10000 [20:48<11:34:58,  4.29s/it]
  3%|▎         | 288/10000 [20:52<11:37:38,  4.31s/it]
  3%|▎         | 289/10000 [20:56<11:31:34,  4.27s/it]
  3%|▎         | 290/10000 [21:01<11:41:07,  4.33s/it]
  3%|▎         | 291/10000 [21:05<11:28:40,  4.26s/it]
  3%|▎         | 292/10000 [21:09<11:33:31,  4.29s/it]
  3%|▎         | 293/10000 [21:14<11:50:02,  4.39s/it]
  3%|▎         | 294/10000 [21:18<11:54:31,  4.42s/it]
  3%|▎         | 295/10000 [21:23<11:51:57,  4.40s/it]
  3%|▎         | 296/10000 [21:27<11:40:35,  4.33s/it]
  3%|▎         | 297/10000 [21:31<11:53:38,  4.41s/it]
  3%|▎         | 298/10000 [21:36<11:46:11,  4.37s/it]
  3%|▎         | 299/10000 [21:40<11:50:49,  4.40s/it]
  3%|▎         | 300/10000 [21:44<11:22:56,  4.22s/it]
                                                      

  3%|▎         | 300/10000 [21:44<11:22:56,  4.22s/it]
  3%|▎         | 301/10000 [21:49<11:46:08,  4.37s/it]
  3%|▎         | 302/10000 [21:53<11:29:45,  4.27s/it]
  3%|▎         | 303/10000 [21:57<11:10:22,  4.15s/it]
  3%|▎         | 304/10000 [22:01<11:42:06,  4.34s/it]
  3%|▎         | 305/10000 [22:05<11:29:37,  4.27s/it]
  3%|▎         | 306/10000 [22:10<11:29:01,  4.26s/it]
  3%|▎         | 307/10000 [22:14<11:22:20,  4.22s/it]
  3%|▎         | 308/10000 [22:19<11:49:26,  4.39s/it]
  3%|▎         | 309/10000 [22:23<11:34:47,  4.30s/it]
  3%|▎         | 310/10000 [22:27<11:26:12,  4.25s/it]
  3%|▎         | 311/10000 [22:31<11:16:07,  4.19s/it]
  3%|▎         | 312/10000 [22:36<11:44:56,  4.37s/it]
  3%|▎         | 313/10000 [22:40<11:44:41,  4.36s/it]
  3%|▎         | 314/10000 [22:44<11:47:34,  4.38s/it]
  3%|▎         | 315/10000 [22:49<11:47:36,  4.38s/it]
  3%|▎         | 316/10000 [22:53<11:40:26,  4.34s/it]
  3%|▎         | 317/10000 [22:57<11:42:33,  4.35s/it]
  3%|▎         | 318/10000 [23:02<11:33:21,  4.30s/it]
  3%|▎         | 319/10000 [23:06<11:44:05,  4.36s/it]
  3%|▎         | 320/10000 [23:11<12:07:50,  4.51s/it]
                                                      

  3%|▎         | 320/10000 [23:11<12:07:50,  4.51s/it]
  3%|▎         | 321/10000 [23:15<11:38:08,  4.33s/it]
  3%|▎         | 322/10000 [23:20<11:59:26,  4.46s/it]
  3%|▎         | 323/10000 [23:24<11:44:12,  4.37s/it]
  3%|▎         | 324/10000 [23:29<12:01:11,  4.47s/it]
  3%|▎         | 325/10000 [23:33<11:52:25,  4.42s/it]
  3%|▎         | 326/10000 [23:37<12:00:51,  4.47s/it]
  3%|▎         | 327/10000 [23:42<11:55:21,  4.44s/it]
  3%|▎         | 328/10000 [23:46<11:46:54,  4.39s/it]
  3%|▎         | 329/10000 [23:51<12:09:00,  4.52s/it]
  3%|▎         | 330/10000 [23:55<11:47:41,  4.39s/it]
  3%|▎         | 331/10000 [23:59<11:40:31,  4.35s/it]
  3%|▎         | 332/10000 [24:04<12:03:48,  4.49s/it]
  3%|▎         | 333/10000 [24:09<12:02:35,  4.48s/it]
  3%|▎         | 334/10000 [24:13<12:04:58,  4.50s/it]
  3%|▎         | 335/10000 [24:18<12:02:02,  4.48s/it]
  3%|▎         | 336/10000 [24:22<12:01:23,  4.48s/it]
  3%|▎         | 337/10000 [24:26<11:31:53,  4.30s/it]
  3%|▎         | 338/10000 [24:30<11:25:13,  4.26s/it]
  3%|▎         | 339/10000 [24:34<11:33:33,  4.31s/it]
  3%|▎         | 340/10000 [24:39<11:29:39,  4.28s/it]
                                                      

  3%|▎         | 340/10000 [24:39<11:29:39,  4.28s/it]
  3%|▎         | 341/10000 [24:43<11:13:28,  4.18s/it]
  3%|▎         | 342/10000 [24:46<10:57:33,  4.09s/it]
  3%|▎         | 343/10000 [24:51<11:16:10,  4.20s/it]
  3%|▎         | 344/10000 [24:55<11:06:12,  4.14s/it]
  3%|▎         | 345/10000 [24:59<11:17:20,  4.21s/it]
  3%|▎         | 346/10000 [25:04<11:45:47,  4.39s/it]
  3%|▎         | 347/10000 [25:08<11:22:16,  4.24s/it]
  3%|▎         | 348/10000 [25:12<11:25:10,  4.26s/it]
  3%|▎         | 349/10000 [25:16<11:14:40,  4.19s/it]
  4%|▎         | 350/10000 [25:21<11:21:40,  4.24s/it]
  4%|▎         | 351/10000 [25:25<11:26:50,  4.27s/it]
  4%|▎         | 352/10000 [25:29<11:19:02,  4.22s/it]
  4%|▎         | 353/10000 [25:34<11:50:38,  4.42s/it]
  4%|▎         | 354/10000 [25:38<11:38:43,  4.35s/it]
  4%|▎         | 355/10000 [25:43<11:46:09,  4.39s/it]
  4%|▎         | 356/10000 [25:48<12:06:20,  4.52s/it]
  4%|▎         | 357/10000 [25:52<12:00:11,  4.48s/it]
  4%|▎         | 358/10000 [25:56<11:45:45,  4.39s/it]
  4%|▎         | 359/10000 [26:00<11:31:11,  4.30s/it]
  4%|▎         | 360/10000 [26:05<11:45:27,  4.39s/it]
                                                      

  4%|▎         | 360/10000 [26:05<11:45:27,  4.39s/it]
  4%|▎         | 361/10000 [26:09<11:47:30,  4.40s/it]
  4%|▎         | 362/10000 [26:13<11:21:13,  4.24s/it]
  4%|▎         | 363/10000 [26:17<11:04:46,  4.14s/it]
  4%|▎         | 364/10000 [26:21<11:15:56,  4.21s/it]
  4%|▎         | 365/10000 [26:26<11:30:23,  4.30s/it]
  4%|▎         | 366/10000 [26:30<11:18:45,  4.23s/it]
  4%|▎         | 367/10000 [26:34<11:17:08,  4.22s/it]
  4%|▎         | 368/10000 [26:39<11:35:48,  4.33s/it]
  4%|▎         | 369/10000 [26:43<11:29:00,  4.29s/it]
  4%|▎         | 370/10000 [26:47<11:27:50,  4.29s/it]
  4%|▎         | 371/10000 [26:52<11:29:41,  4.30s/it]
  4%|▎         | 372/10000 [26:56<11:46:09,  4.40s/it]
  4%|▎         | 373/10000 [27:00<11:36:04,  4.34s/it]
  4%|▎         | 374/10000 [27:04<11:22:49,  4.26s/it]
  4%|▍         | 375/10000 [27:09<11:30:14,  4.30s/it]
  4%|▍         | 376/10000 [27:13<11:25:11,  4.27s/it]
  4%|▍         | 377/10000 [27:17<11:20:09,  4.24s/it]
  4%|▍         | 378/10000 [27:22<11:43:35,  4.39s/it]
  4%|▍         | 379/10000 [27:26<11:28:31,  4.29s/it]
  4%|▍         | 380/10000 [27:30<11:21:09,  4.25s/it]
                                                      
{'loss': '0.0275', 'grad_norm': '2.938', 'learning_rate': '4.9e-06', 'num_tokens': '5.901e+04', 'completions/mean_length': '4.356', 'completions/min_length': '3.7', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.356', 'completions/min_terminated_length': '3.7', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3797', 'rewards/reward_semantic/std': '0.1427', 'rewards/reward_length/mean': '-0.04658', 'rewards/reward_length/std': '0.008723', 'reward': '1.112', 'reward_std': '0.04278', 'frac_reward_zero_std': '0.25', 'kl': '0.04557', 'entropy': '0.5731', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.26', 'epoch': '0.03281'}
{'loss': '0.03603', 'grad_norm': '6.45', 'learning_rate': '4.891e-06', 'num_tokens': '6.522e+04', 'completions/mean_length': '5.144', 'completions/min_length': '4.05', 'completions/max_length': '6.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.144', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '6.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4297', 'rewards/reward_semantic/std': '0.08431', 'rewards/reward_length/mean': '-0.06016', 'rewards/reward_length/std': '0.01355', 'reward': '1.126', 'reward_std': '0.02571', 'frac_reward_zero_std': '0.4', 'kl': '0.06052', 'entropy': '0.595', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.38', 'epoch': '0.03609'}
{'loss': '0.05899', 'grad_norm': '6.051', 'learning_rate': '4.881e-06', 'num_tokens': '7.133e+04', 'completions/mean_length': '4.456', 'completions/min_length': '3.4', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.456', 'completions/min_terminated_length': '3.4', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.486', 'rewards/reward_semantic/std': '0.1516', 'rewards/reward_length/mean': '-0.06814', 'rewards/reward_length/std': '0.01963', 'reward': '1.142', 'reward_std': '0.046', 'frac_reward_zero_std': '0.3', 'kl': '0.02611', 'entropy': '0.5288', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.285', 'epoch': '0.03937'}
{'loss': '0.1343', 'grad_norm': '13.39', 'learning_rate': '4.871e-06', 'num_tokens': '7.755e+04', 'completions/mean_length': '4.65', 'completions/min_length': '3.25', 'completions/max_length': '8.75', 'completions/clipped_ratio': '0.00625', 'completions/mean_terminated_length': '4.283', 'completions/min_terminated_length': '3.25', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2612', 'rewards/reward_semantic/std': '0.05506', 'rewards/reward_length/mean': '-0.05339', 'rewards/reward_length/std': '0.0187', 'reward': '1.076', 'reward_std': '0.01715', 'frac_reward_zero_std': '0.2', 'kl': '0.07412', 'entropy': '0.6873', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.516', 'epoch': '0.04265'}
{'loss': '0.02459', 'grad_norm': '4.261', 'learning_rate': '4.861e-06', 'num_tokens': '8.347e+04', 'completions/mean_length': '4.544', 'completions/min_length': '3.65', 'completions/max_length': '5.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.544', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '5.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3644', 'rewards/reward_semantic/std': '0.09163', 'rewards/reward_length/mean': '-0.05837', 'rewards/reward_length/std': '0.01181', 'reward': '1.106', 'reward_std': '0.02763', 'frac_reward_zero_std': '0.35', 'kl': '0.1104', 'entropy': '0.5345', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.288', 'epoch': '0.04593'}
{'loss': '0.02182', 'grad_norm': '0.008741', 'learning_rate': '4.851e-06', 'num_tokens': '8.965e+04', 'completions/mean_length': '4.237', 'completions/min_length': '2.95', 'completions/max_length': '5.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.237', 'completions/min_terminated_length': '2.95', 'completions/max_terminated_length': '5.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3463', 'rewards/reward_semantic/std': '0.12', 'rewards/reward_length/mean': '-0.05469', 'rewards/reward_length/std': '0.01777', 'reward': '1.101', 'reward_std': '0.03634', 'frac_reward_zero_std': '0.2', 'kl': '0.1457', 'entropy': '0.7965', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.288', 'epoch': '0.04921'}
{'loss': '0.05665', 'grad_norm': '4.915', 'learning_rate': '4.841e-06', 'num_tokens': '9.523e+04', 'completions/mean_length': '4.531', 'completions/min_length': '3.45', 'completions/max_length': '6.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.531', 'completions/min_terminated_length': '3.45', 'completions/max_terminated_length': '6.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2358', 'rewards/reward_semantic/std': '0.09729', 'rewards/reward_length/mean': '-0.05632', 'rewards/reward_length/std': '0.01544', 'reward': '1.068', 'reward_std': '0.02938', 'frac_reward_zero_std': '0.2', 'kl': '0.1062', 'entropy': '0.6368', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.332', 'epoch': '0.05249'}
{'loss': '0.06411', 'grad_norm': '7.897', 'learning_rate': '4.831e-06', 'num_tokens': '1.009e+05', 'completions/mean_length': '4.606', 'completions/min_length': '3.6', 'completions/max_length': '6.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.606', 'completions/min_terminated_length': '3.6', 'completions/max_terminated_length': '6.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3308', 'rewards/reward_semantic/std': '0.06247', 'rewards/reward_length/mean': '-0.05127', 'rewards/reward_length/std': '0.01417', 'reward': '1.097', 'reward_std': '0.01919', 'frac_reward_zero_std': '0.25', 'kl': '0.2204', 'entropy': '0.7942', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.363', 'epoch': '0.05577'}
{'loss': '0.01024', 'grad_norm': '0.02093', 'learning_rate': '4.82e-06', 'num_tokens': '1.068e+05', 'completions/mean_length': '4.569', 'completions/min_length': '3.7', 'completions/max_length': '5.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.569', 'completions/min_terminated_length': '3.7', 'completions/max_terminated_length': '5.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3768', 'rewards/reward_semantic/std': '0.05658', 'rewards/reward_length/mean': '-0.05686', 'rewards/reward_length/std': '0.008187', 'reward': '1.11', 'reward_std': '0.01714', 'frac_reward_zero_std': '0.4', 'kl': '0.07341', 'entropy': '0.4296', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.287', 'epoch': '0.05906'}

  4%|▍         | 380/10000 [27:30<11:21:09,  4.25s/it]
  4%|▍         | 381/10000 [27:34<11:12:54,  4.20s/it]
  4%|▍         | 382/10000 [27:39<11:28:07,  4.29s/it]
  4%|▍         | 383/10000 [27:43<11:31:28,  4.31s/it]
  4%|▍         | 384/10000 [27:48<11:37:26,  4.35s/it]
  4%|▍         | 385/10000 [27:52<11:44:18,  4.40s/it]
  4%|▍         | 386/10000 [27:56<11:30:39,  4.31s/it]
  4%|▍         | 387/10000 [28:01<11:35:49,  4.34s/it]
  4%|▍         | 388/10000 [28:05<11:18:08,  4.23s/it]
  4%|▍         | 389/10000 [28:09<11:28:22,  4.30s/it]
  4%|▍         | 390/10000 [28:13<11:18:19,  4.24s/it]
  4%|▍         | 391/10000 [28:18<11:33:43,  4.33s/it]
  4%|▍         | 392/10000 [28:22<11:39:57,  4.37s/it]
  4%|▍         | 393/10000 [28:26<11:30:25,  4.31s/it]
  4%|▍         | 394/10000 [28:30<11:09:29,  4.18s/it]
  4%|▍         | 395/10000 [28:35<11:25:22,  4.28s/it]
  4%|▍         | 396/10000 [28:39<11:43:57,  4.40s/it]
  4%|▍         | 397/10000 [28:43<11:26:48,  4.29s/it]
  4%|▍         | 398/10000 [28:47<11:11:25,  4.20s/it]
  4%|▍         | 399/10000 [28:52<11:35:14,  4.34s/it]
  4%|▍         | 400/10000 [28:56<11:13:32,  4.21s/it]
                                                      

  4%|▍         | 400/10000 [28:56<11:13:32,  4.21s/it]
  4%|▍         | 401/10000 [29:00<11:14:22,  4.22s/it]
  4%|▍         | 402/10000 [29:05<11:23:58,  4.28s/it]
  4%|▍         | 403/10000 [29:09<11:25:38,  4.29s/it]
  4%|▍         | 404/10000 [29:13<11:09:55,  4.19s/it]
  4%|▍         | 405/10000 [29:17<11:08:59,  4.18s/it]
  4%|▍         | 406/10000 [29:21<11:20:54,  4.26s/it]
  4%|▍         | 407/10000 [29:25<11:03:09,  4.15s/it]
  4%|▍         | 408/10000 [29:29<10:51:06,  4.07s/it]
  4%|▍         | 409/10000 [29:34<11:13:26,  4.21s/it]
  4%|▍         | 410/10000 [29:38<11:20:49,  4.26s/it]
  4%|▍         | 411/10000 [29:42<11:18:24,  4.24s/it]
  4%|▍         | 412/10000 [29:47<11:14:20,  4.22s/it]
  4%|▍         | 413/10000 [29:52<11:53:28,  4.47s/it]
  4%|▍         | 414/10000 [29:56<11:28:47,  4.31s/it]
  4%|▍         | 415/10000 [30:00<11:39:00,  4.38s/it]
  4%|▍         | 416/10000 [30:05<12:01:38,  4.52s/it]
  4%|▍         | 417/10000 [30:09<11:44:34,  4.41s/it]
  4%|▍         | 418/10000 [30:13<11:19:14,  4.25s/it]
  4%|▍         | 419/10000 [30:17<11:20:52,  4.26s/it]
  4%|▍         | 420/10000 [30:22<11:54:24,  4.47s/it]
                                                      

  4%|▍         | 420/10000 [30:22<11:54:24,  4.47s/it]
  4%|▍         | 421/10000 [30:26<11:43:52,  4.41s/it]
  4%|▍         | 422/10000 [30:31<11:29:59,  4.32s/it]
  4%|▍         | 423/10000 [30:35<11:36:02,  4.36s/it]
  4%|▍         | 424/10000 [30:39<11:32:46,  4.34s/it]
  4%|▍         | 425/10000 [30:43<11:21:15,  4.27s/it]
  4%|▍         | 426/10000 [30:48<11:16:00,  4.24s/it]
  4%|▍         | 427/10000 [30:52<11:33:34,  4.35s/it]
  4%|▍         | 428/10000 [30:56<11:19:30,  4.26s/it]
  4%|▍         | 429/10000 [31:00<11:11:01,  4.21s/it]
  4%|▍         | 430/10000 [31:05<11:49:09,  4.45s/it]
  4%|▍         | 431/10000 [31:10<11:44:23,  4.42s/it]
  4%|▍         | 432/10000 [31:14<11:33:57,  4.35s/it]
  4%|▍         | 433/10000 [31:18<11:28:34,  4.32s/it]
  4%|▍         | 434/10000 [31:23<11:32:03,  4.34s/it]
  4%|▍         | 435/10000 [31:27<11:50:56,  4.46s/it]
  4%|▍         | 436/10000 [31:32<11:54:44,  4.48s/it]
  4%|▍         | 437/10000 [31:36<11:45:56,  4.43s/it]
  4%|▍         | 438/10000 [31:40<11:35:32,  4.36s/it]
  4%|▍         | 439/10000 [31:45<11:40:26,  4.40s/it]
  4%|▍         | 440/10000 [31:49<11:17:39,  4.25s/it]
                                                      

  4%|▍         | 440/10000 [31:49<11:17:39,  4.25s/it]
  4%|▍         | 441/10000 [31:54<12:13:15,  4.60s/it]
  4%|▍         | 442/10000 [31:58<11:44:03,  4.42s/it]
  4%|▍         | 443/10000 [32:02<11:19:48,  4.27s/it]
  4%|▍         | 444/10000 [32:07<11:40:34,  4.40s/it]
  4%|▍         | 445/10000 [32:11<11:33:32,  4.36s/it]
  4%|▍         | 446/10000 [32:15<11:18:48,  4.26s/it]
  4%|▍         | 447/10000 [32:19<11:09:43,  4.21s/it]
  4%|▍         | 448/10000 [32:24<11:27:29,  4.32s/it]
  4%|▍         | 449/10000 [32:28<11:09:57,  4.21s/it]
  4%|▍         | 450/10000 [32:32<11:13:19,  4.23s/it]
  5%|▍         | 451/10000 [32:37<11:45:30,  4.43s/it]
  5%|▍         | 452/10000 [32:41<11:43:45,  4.42s/it]
  5%|▍         | 453/10000 [32:46<11:43:24,  4.42s/it]
  5%|▍         | 454/10000 [32:50<11:32:59,  4.36s/it]
  5%|▍         | 455/10000 [32:54<11:40:55,  4.41s/it]
  5%|▍         | 456/10000 [32:59<11:44:16,  4.43s/it]
  5%|▍         | 457/10000 [33:03<11:48:02,  4.45s/it]
  5%|▍         | 458/10000 [33:08<11:57:26,  4.51s/it]
  5%|▍         | 459/10000 [33:13<12:08:16,  4.58s/it]
  5%|▍         | 460/10000 [33:17<11:48:36,  4.46s/it]
                                                      

  5%|▍         | 460/10000 [33:17<11:48:36,  4.46s/it]
  5%|▍         | 461/10000 [33:21<11:42:53,  4.42s/it]
  5%|▍         | 462/10000 [33:25<11:31:21,  4.35s/it]
  5%|▍         | 463/10000 [33:30<11:18:54,  4.27s/it]
  5%|▍         | 464/10000 [33:34<11:24:33,  4.31s/it]
  5%|▍         | 465/10000 [33:38<11:21:10,  4.29s/it]
  5%|▍         | 466/10000 [33:43<11:32:10,  4.36s/it]
  5%|▍         | 467/10000 [33:47<11:27:15,  4.33s/it]
  5%|▍         | 468/10000 [33:51<11:32:00,  4.36s/it]
  5%|▍         | 469/10000 [33:56<11:36:23,  4.38s/it]
  5%|▍         | 470/10000 [34:00<11:37:17,  4.39s/it]
  5%|▍         | 471/10000 [34:04<11:30:45,  4.35s/it]
  5%|▍         | 472/10000 [34:09<11:28:08,  4.33s/it]
  5%|▍         | 473/10000 [34:13<11:18:32,  4.27s/it]
  5%|▍         | 474/10000 [34:17<11:22:04,  4.30s/it]
  5%|▍         | 475/10000 [34:21<11:16:09,  4.26s/it]
  5%|▍         | 476/10000 [34:26<11:38:26,  4.40s/it]
  5%|▍         | 477/10000 [34:31<11:41:25,  4.42s/it]
  5%|▍         | 478/10000 [34:35<11:43:42,  4.43s/it]
  5%|▍         | 479/10000 [34:39<11:40:40,  4.42s/it]
  5%|▍         | 480/10000 [34:43<11:15:51,  4.26s/it]
                                                      

  5%|▍         | 480/10000 [34:43<11:15:51,  4.26s/it]
  5%|▍         | 481/10000 [34:48<11:21:17,  4.29s/it]
  5%|▍         | 482/10000 [34:52<11:22:34,  4.30s/it]
  5%|▍         | 483/10000 [34:56<11:17:27,  4.27s/it]
  5%|▍         | 484/10000 [35:00<11:03:18,  4.18s/it]
  5%|▍         | 485/10000 [35:05<11:08:55,  4.22s/it]
  5%|▍         | 486/10000 [35:09<11:17:43,  4.27s/it]
  5%|▍         | 487/10000 [35:13<11:23:32,  4.31s/it]
  5%|▍         | 488/10000 [35:18<11:23:41,  4.31s/it]
  5%|▍         | 489/10000 [35:22<11:17:15,  4.27s/it]
  5%|▍         | 490/10000 [35:26<11:19:39,  4.29s/it]
  5%|▍         | 491/10000 [35:30<11:18:42,  4.28s/it]
  5%|▍         | 492/10000 [35:35<11:16:04,  4.27s/it]
  5%|▍         | 493/10000 [35:39<11:40:06,  4.42s/it]
  5%|▍         | 494/10000 [35:44<11:32:52,  4.37s/it]
  5%|▍         | 495/10000 [35:48<11:29:46,  4.35s/it]
  5%|▍         | 496/10000 [35:52<11:28:53,  4.35s/it]
  5%|▍         | 497/10000 [35:57<11:40:34,  4.42s/it]
  5%|▍         | 498/10000 [36:01<11:31:23,  4.37s/it]
  5%|▍         | 499/10000 [36:05<11:24:16,  4.32s/it]
  5%|▌         | 500/10000 [36:10<11:27:26,  4.34s/it]
                                                      

  5%|▌         | 500/10000 [36:10<11:27:26,  4.34s/it]
  5%|▌         | 501/10000 [36:15<11:58:11,  4.54s/it]
  5%|▌         | 502/10000 [36:19<11:47:20,  4.47s/it]
  5%|▌         | 503/10000 [36:23<11:34:01,  4.38s/it]
  5%|▌         | 504/10000 [36:28<11:31:09,  4.37s/it]
  5%|▌         | 505/10000 [36:32<11:43:26,  4.45s/it]
  5%|▌         | 506/10000 [36:36<11:34:33,  4.39s/it]
  5%|▌         | 507/10000 [36:41<11:55:09,  4.52s/it]
  5%|▌         | 508/10000 [36:45<11:31:23,  4.37s/it]
  5%|▌         | 509/10000 [36:49<11:14:07,  4.26s/it]
  5%|▌         | 510/10000 [36:54<11:11:22,  4.24s/it]
  5%|▌         | 511/10000 [36:58<11:09:25,  4.23s/it]
  5%|▌         | 512/10000 [37:02<11:04:53,  4.20s/it]
  5%|▌         | 513/10000 [37:06<10:58:03,  4.16s/it]
  5%|▌         | 514/10000 [37:10<11:09:47,  4.24s/it]
  5%|▌         | 515/10000 [37:15<11:09:00,  4.23s/it]
  5%|▌         | 516/10000 [37:18<10:54:07,  4.14s/it]
  5%|▌         | 517/10000 [37:23<11:09:43,  4.24s/it]
  5%|▌         | 518/10000 [37:27<11:11:24,  4.25s/it]
  5%|▌         | 519/10000 [37:32<11:14:13,  4.27s/it]
  5%|▌         | 520/10000 [37:35<10:57:17,  4.16s/it]
                                                      

  5%|▌         | 520/10000 [37:35<10:57:17,  4.16s/it]
  5%|▌         | 521/10000 [37:40<10:58:30,  4.17s/it]
  5%|▌         | 522/10000 [37:44<10:59:35,  4.18s/it]
  5%|▌         | 523/10000 [37:48<11:09:33,  4.24s/it]
  5%|▌         | 524/10000 [37:53<11:16:43,  4.28s/it]
  5%|▌         | 525/10000 [37:57<11:41:50,  4.44s/it]
  5%|▌         | 526/10000 [38:02<11:35:56,  4.41s/it]
  5%|▌         | 527/10000 [38:06<11:11:46,  4.25s/it]
  5%|▌         | 528/10000 [38:10<11:22:59,  4.33s/it]
  5%|▌         | 529/10000 [38:14<11:13:13,  4.27s/it]
  5%|▌         | 530/10000 [38:19<11:29:00,  4.37s/it]
  5%|▌         | 531/10000 [38:23<11:19:46,  4.31s/it]
  5%|▌         | 532/10000 [38:27<11:13:17,  4.27s/it]
  5%|▌         | 533/10000 [38:31<11:08:06,  4.23s/it]
  5%|▌         | 534/10000 [38:35<10:52:00,  4.13s/it]
  5%|▌         | 535/10000 [38:40<11:16:41,  4.29s/it]
  5%|▌         | 536/10000 [38:44<11:30:04,  4.37s/it]
  5%|▌         | 537/10000 [38:49<11:27:10,  4.36s/it]
  5%|▌         | 538/10000 [38:53<11:39:38,  4.44s/it]
  5%|▌         | 539/10000 [38:58<11:27:03,  4.36s/it]
  5%|▌         | 540/10000 [39:02<11:24:56,  4.34s/it]
                                                      

  5%|▌         | 540/10000 [39:02<11:24:56,  4.34s/it]
  5%|▌         | 541/10000 [39:06<11:16:44,  4.29s/it]
  5%|▌         | 542/10000 [39:11<11:26:46,  4.36s/it]
  5%|▌         | 543/10000 [39:14<11:04:53,  4.22s/it]
  5%|▌         | 544/10000 [39:19<10:56:04,  4.16s/it]
  5%|▌         | 545/10000 [39:23<10:58:27,  4.18s/it]
  5%|▌         | 546/10000 [39:27<10:57:39,  4.17s/it]
  5%|▌         | 547/10000 [39:31<11:02:04,  4.20s/it]
  5%|▌         | 548/10000 [39:35<11:00:23,  4.19s/it]
  5%|▌         | 549/10000 [39:40<11:15:20,  4.29s/it]
  6%|▌         | 550/10000 [39:44<11:06:58,  4.23s/it]
  6%|▌         | 551/10000 [39:48<10:50:47,  4.13s/it]
  6%|▌         | 552/10000 [39:52<11:08:20,  4.24s/it]
  6%|▌         | 553/10000 [39:57<11:06:08,  4.23s/it]
  6%|▌         | 554/10000 [40:01<11:06:57,  4.24s/it]
  6%|▌         | 555/10000 [40:05<11:07:54,  4.24s/it]
  6%|▌         | 556/10000 [40:09<11:16:11,  4.30s/it]
  6%|▌         | 557/10000 [40:14<11:14:48,  4.29s/it]
  6%|▌         | 558/10000 [40:18<11:01:55,  4.21s/it]
  6%|▌         | 559/10000 [40:22<11:03:10,  4.21s/it]
  6%|▌         | 560/10000 [40:27<11:22:40,  4.34s/it]
                                                      
{'loss': '-0.01211', 'grad_norm': '0.05952', 'learning_rate': '4.81e-06', 'num_tokens': '1.126e+05', 'completions/mean_length': '4.725', 'completions/min_length': '3.75', 'completions/max_length': '5.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.725', 'completions/min_terminated_length': '3.75', 'completions/max_terminated_length': '5.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3539', 'rewards/reward_semantic/std': '0.09912', 'rewards/reward_length/mean': '-0.05332', 'rewards/reward_length/std': '0.009659', 'reward': '1.103', 'reward_std': '0.02983', 'frac_reward_zero_std': '0.3', 'kl': '0.1093', 'entropy': '0.5075', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.248', 'epoch': '0.06234'}
{'loss': '0.02361', 'grad_norm': '0.1114', 'learning_rate': '4.801e-06', 'num_tokens': '1.183e+05', 'completions/mean_length': '4.281', 'completions/min_length': '3.25', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.281', 'completions/min_terminated_length': '3.25', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4171', 'rewards/reward_semantic/std': '0.1745', 'rewards/reward_length/mean': '-0.05713', 'rewards/reward_length/std': '0.01449', 'reward': '1.122', 'reward_std': '0.0525', 'frac_reward_zero_std': '0.2', 'kl': '0.1403', 'entropy': '0.6337', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.27', 'epoch': '0.06562'}
{'loss': '0.06649', 'grad_norm': '5.328', 'learning_rate': '4.791e-06', 'num_tokens': '1.241e+05', 'completions/mean_length': '4.394', 'completions/min_length': '3.45', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.394', 'completions/min_terminated_length': '3.45', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3078', 'rewards/reward_semantic/std': '0.0711', 'rewards/reward_length/mean': '-0.05554', 'rewards/reward_length/std': '0.01265', 'reward': '1.09', 'reward_std': '0.02168', 'frac_reward_zero_std': '0.3', 'kl': '0.09478', 'entropy': '0.5983', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.293', 'epoch': '0.0689'}
{'loss': '0.0713', 'grad_norm': '0.2607', 'learning_rate': '4.781e-06', 'num_tokens': '1.305e+05', 'completions/mean_length': '4.75', 'completions/min_length': '3.45', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.75', 'completions/min_terminated_length': '3.45', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4138', 'rewards/reward_semantic/std': '0.0793', 'rewards/reward_length/mean': '-0.05481', 'rewards/reward_length/std': '0.01224', 'reward': '1.121', 'reward_std': '0.02405', 'frac_reward_zero_std': '0.3', 'kl': '0.1251', 'entropy': '0.5342', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.305', 'epoch': '0.07218'}
{'loss': '0.09641', 'grad_norm': '3.259', 'learning_rate': '4.771e-06', 'num_tokens': '1.363e+05', 'completions/mean_length': '4.862', 'completions/min_length': '3.5', 'completions/max_length': '6.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.862', 'completions/min_terminated_length': '3.5', 'completions/max_terminated_length': '6.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2071', 'rewards/reward_semantic/std': '0.1539', 'rewards/reward_length/mean': '-0.06187', 'rewards/reward_length/std': '0.01569', 'reward': '1.059', 'reward_std': '0.04661', 'frac_reward_zero_std': '0.1', 'kl': '0.1585', 'entropy': '0.693', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.391', 'epoch': '0.07546'}
{'loss': '0.1074', 'grad_norm': '0.0372', 'learning_rate': '4.761e-06', 'num_tokens': '1.423e+05', 'completions/mean_length': '4.469', 'completions/min_length': '3.55', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '3.55', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.368', 'rewards/reward_semantic/std': '0.1292', 'rewards/reward_length/mean': '-0.04971', 'rewards/reward_length/std': '0.01197', 'reward': '1.108', 'reward_std': '0.03923', 'frac_reward_zero_std': '0.25', 'kl': '0.1856', 'entropy': '0.5916', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.301', 'epoch': '0.07874'}
{'loss': '0.05171', 'grad_norm': '0.01287', 'learning_rate': '4.751e-06', 'num_tokens': '1.483e+05', 'completions/mean_length': '4.612', 'completions/min_length': '3.7', 'completions/max_length': '5.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.612', 'completions/min_terminated_length': '3.7', 'completions/max_terminated_length': '5.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2118', 'rewards/reward_semantic/std': '0.1037', 'rewards/reward_length/mean': '-0.05286', 'rewards/reward_length/std': '0.0111', 'reward': '1.061', 'reward_std': '0.03139', 'frac_reward_zero_std': '0.2', 'kl': '0.235', 'entropy': '0.6024', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.301', 'epoch': '0.08202'}
{'loss': '0.0233', 'grad_norm': '0.01736', 'learning_rate': '4.741e-06', 'num_tokens': '1.54e+05', 'completions/mean_length': '3.806', 'completions/min_length': '3.2', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '3.806', 'completions/min_terminated_length': '3.2', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4632', 'rewards/reward_semantic/std': '0.1097', 'rewards/reward_length/mean': '-0.05071', 'rewards/reward_length/std': '0.007349', 'reward': '1.136', 'reward_std': '0.03298', 'frac_reward_zero_std': '0.3', 'kl': '0.1286', 'entropy': '0.4171', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.211', 'epoch': '0.0853'}
{'loss': '0.05552', 'grad_norm': '3.972', 'learning_rate': '4.731e-06', 'num_tokens': '1.599e+05', 'completions/mean_length': '4.519', 'completions/min_length': '3.45', 'completions/max_length': '5.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.519', 'completions/min_terminated_length': '3.45', 'completions/max_terminated_length': '5.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3229', 'rewards/reward_semantic/std': '0.03359', 'rewards/reward_length/mean': '-0.04285', 'rewards/reward_length/std': '0.008181', 'reward': '1.095', 'reward_std': '0.01039', 'frac_reward_zero_std': '0.35', 'kl': '0.3025', 'entropy': '0.688', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.303', 'epoch': '0.08858'}

  6%|▌         | 560/10000 [40:27<11:22:40,  4.34s/it]
  6%|▌         | 561/10000 [40:31<11:24:41,  4.35s/it]
  6%|▌         | 562/10000 [40:35<11:03:04,  4.22s/it]
  6%|▌         | 563/10000 [40:39<11:12:12,  4.27s/it]
  6%|▌         | 564/10000 [40:44<11:13:25,  4.28s/it]
  6%|▌         | 565/10000 [40:48<11:12:06,  4.27s/it]
  6%|▌         | 566/10000 [40:52<11:02:05,  4.21s/it]
  6%|▌         | 567/10000 [40:56<11:08:45,  4.25s/it]
  6%|▌         | 568/10000 [41:00<10:59:11,  4.19s/it]
  6%|▌         | 569/10000 [41:05<11:01:18,  4.21s/it]
  6%|▌         | 570/10000 [41:09<10:56:46,  4.18s/it]
  6%|▌         | 571/10000 [41:13<10:59:40,  4.20s/it]
  6%|▌         | 572/10000 [41:17<11:01:28,  4.21s/it]
  6%|▌         | 573/10000 [41:21<10:52:58,  4.16s/it]
  6%|▌         | 574/10000 [41:25<10:52:41,  4.15s/it]
  6%|▌         | 575/10000 [41:29<10:48:09,  4.13s/it]
  6%|▌         | 576/10000 [41:33<10:32:28,  4.03s/it]
  6%|▌         | 577/10000 [41:37<10:34:10,  4.04s/it]
  6%|▌         | 578/10000 [41:41<10:37:34,  4.06s/it]
  6%|▌         | 579/10000 [41:45<10:36:49,  4.06s/it]
  6%|▌         | 580/10000 [41:50<10:49:33,  4.14s/it]
                                                      

  6%|▌         | 580/10000 [41:50<10:49:33,  4.14s/it]
  6%|▌         | 581/10000 [41:54<11:03:48,  4.23s/it]
  6%|▌         | 582/10000 [41:58<10:56:50,  4.18s/it]
  6%|▌         | 583/10000 [42:03<11:07:26,  4.25s/it]
  6%|▌         | 584/10000 [42:07<11:08:28,  4.26s/it]
  6%|▌         | 585/10000 [42:11<10:47:08,  4.12s/it]
  6%|▌         | 586/10000 [42:15<10:46:48,  4.12s/it]
  6%|▌         | 587/10000 [42:19<10:40:46,  4.08s/it]
  6%|▌         | 588/10000 [42:24<11:13:19,  4.29s/it]
  6%|▌         | 589/10000 [42:28<11:09:00,  4.27s/it]
  6%|▌         | 590/10000 [42:33<11:32:24,  4.41s/it]
  6%|▌         | 591/10000 [42:37<11:18:03,  4.32s/it]
  6%|▌         | 592/10000 [42:41<11:05:46,  4.25s/it]
  6%|▌         | 593/10000 [42:45<10:57:49,  4.20s/it]
  6%|▌         | 594/10000 [42:49<10:46:35,  4.12s/it]
  6%|▌         | 595/10000 [42:53<11:07:08,  4.26s/it]
  6%|▌         | 596/10000 [42:58<11:07:21,  4.26s/it]
  6%|▌         | 597/10000 [43:02<11:07:50,  4.26s/it]
  6%|▌         | 598/10000 [43:06<11:12:28,  4.29s/it]
  6%|▌         | 599/10000 [43:10<10:57:00,  4.19s/it]
  6%|▌         | 600/10000 [43:14<10:57:16,  4.20s/it]
                                                      

  6%|▌         | 600/10000 [43:14<10:57:16,  4.20s/it]
  6%|▌         | 601/10000 [43:19<10:56:26,  4.19s/it]
  6%|▌         | 602/10000 [43:23<11:16:39,  4.32s/it]
  6%|▌         | 603/10000 [43:27<10:58:01,  4.20s/it]
  6%|▌         | 604/10000 [43:31<10:49:44,  4.15s/it]
  6%|▌         | 605/10000 [43:36<11:01:45,  4.23s/it]
  6%|▌         | 606/10000 [43:40<10:50:02,  4.15s/it]
  6%|▌         | 607/10000 [43:44<10:49:11,  4.15s/it]
  6%|▌         | 608/10000 [43:48<10:35:18,  4.06s/it]
  6%|▌         | 609/10000 [43:52<10:39:38,  4.09s/it]
  6%|▌         | 610/10000 [43:56<10:29:57,  4.03s/it]
  6%|▌         | 611/10000 [44:00<10:31:25,  4.04s/it]
  6%|▌         | 612/10000 [44:04<10:48:16,  4.14s/it]
  6%|▌         | 613/10000 [44:08<10:49:14,  4.15s/it]
  6%|▌         | 614/10000 [44:12<10:35:13,  4.06s/it]
  6%|▌         | 615/10000 [44:16<10:35:22,  4.06s/it]
  6%|▌         | 616/10000 [44:21<10:58:32,  4.21s/it]
  6%|▌         | 617/10000 [44:25<10:52:42,  4.17s/it]
  6%|▌         | 618/10000 [44:29<10:56:25,  4.20s/it]
  6%|▌         | 619/10000 [44:33<11:04:08,  4.25s/it]
  6%|▌         | 620/10000 [44:38<10:55:38,  4.19s/it]
                                                      

  6%|▌         | 620/10000 [44:38<10:55:38,  4.19s/it]
  6%|▌         | 621/10000 [44:42<10:49:31,  4.16s/it]
  6%|▌         | 622/10000 [44:46<10:39:08,  4.09s/it]
  6%|▌         | 623/10000 [44:50<10:55:04,  4.19s/it]
  6%|▌         | 624/10000 [44:54<11:05:31,  4.26s/it]
  6%|▋         | 625/10000 [44:58<10:57:48,  4.21s/it]
  6%|▋         | 626/10000 [45:03<11:06:26,  4.27s/it]
  6%|▋         | 627/10000 [45:07<10:57:13,  4.21s/it]
  6%|▋         | 628/10000 [45:11<11:13:49,  4.31s/it]
  6%|▋         | 629/10000 [45:16<11:02:38,  4.24s/it]
  6%|▋         | 630/10000 [45:20<11:12:19,  4.31s/it]
  6%|▋         | 631/10000 [45:24<11:02:13,  4.24s/it]
  6%|▋         | 632/10000 [45:28<10:56:59,  4.21s/it]
  6%|▋         | 633/10000 [45:33<11:01:09,  4.24s/it]
  6%|▋         | 634/10000 [45:37<10:53:45,  4.19s/it]
  6%|▋         | 635/10000 [45:41<10:49:13,  4.16s/it]
  6%|▋         | 636/10000 [45:45<10:58:46,  4.22s/it]
  6%|▋         | 637/10000 [45:49<11:04:12,  4.26s/it]
  6%|▋         | 638/10000 [45:53<10:45:47,  4.14s/it]
  6%|▋         | 639/10000 [45:57<10:41:49,  4.11s/it]
  6%|▋         | 640/10000 [46:01<10:44:21,  4.13s/it]
                                                      

  6%|▋         | 640/10000 [46:01<10:44:21,  4.13s/it]
  6%|▋         | 641/10000 [46:06<10:42:05,  4.12s/it]
  6%|▋         | 642/10000 [46:10<10:40:27,  4.11s/it]
  6%|▋         | 643/10000 [46:14<10:42:28,  4.12s/it]
  6%|▋         | 644/10000 [46:18<10:49:50,  4.17s/it]
  6%|▋         | 645/10000 [46:22<10:31:46,  4.05s/it]
  6%|▋         | 646/10000 [46:26<10:32:05,  4.05s/it]
  6%|▋         | 647/10000 [46:30<10:46:17,  4.15s/it]
  6%|▋         | 648/10000 [46:34<10:43:03,  4.13s/it]
  6%|▋         | 649/10000 [46:39<10:45:32,  4.14s/it]
  6%|▋         | 650/10000 [46:43<10:42:44,  4.12s/it]
  7%|▋         | 651/10000 [46:47<10:59:28,  4.23s/it]
  7%|▋         | 652/10000 [46:51<10:51:58,  4.18s/it]
  7%|▋         | 653/10000 [46:55<10:49:52,  4.17s/it]
  7%|▋         | 654/10000 [46:59<10:45:30,  4.14s/it]
  7%|▋         | 655/10000 [47:03<10:30:10,  4.05s/it]
  7%|▋         | 656/10000 [47:07<10:31:47,  4.06s/it]
  7%|▋         | 657/10000 [47:11<10:30:14,  4.05s/it]
  7%|▋         | 658/10000 [47:16<10:46:58,  4.16s/it]
  7%|▋         | 659/10000 [47:20<10:42:36,  4.13s/it]
  7%|▋         | 660/10000 [47:24<10:26:28,  4.02s/it]
                                                      

  7%|▋         | 660/10000 [47:24<10:26:28,  4.02s/it]
  7%|▋         | 661/10000 [47:28<10:42:07,  4.13s/it]
  7%|▋         | 662/10000 [47:32<10:39:57,  4.11s/it]
  7%|▋         | 663/10000 [47:36<10:38:35,  4.10s/it]
  7%|▋         | 664/10000 [47:40<10:40:11,  4.11s/it]
  7%|▋         | 665/10000 [47:45<10:52:57,  4.20s/it]
  7%|▋         | 666/10000 [47:49<10:37:49,  4.10s/it]
  7%|▋         | 667/10000 [47:53<10:42:50,  4.13s/it]
  7%|▋         | 668/10000 [47:57<11:01:22,  4.25s/it]
  7%|▋         | 669/10000 [48:01<10:54:53,  4.21s/it]
  7%|▋         | 670/10000 [48:06<11:00:42,  4.25s/it]
  7%|▋         | 671/10000 [48:09<10:39:19,  4.11s/it]
  7%|▋         | 672/10000 [48:14<10:58:29,  4.24s/it]
  7%|▋         | 673/10000 [48:18<10:50:55,  4.19s/it]
  7%|▋         | 674/10000 [48:22<10:39:14,  4.11s/it]
  7%|▋         | 675/10000 [48:26<10:50:57,  4.19s/it]
  7%|▋         | 676/10000 [48:30<10:45:16,  4.15s/it]
  7%|▋         | 677/10000 [48:35<10:41:18,  4.13s/it]
  7%|▋         | 678/10000 [48:39<10:38:13,  4.11s/it]
  7%|▋         | 679/10000 [48:43<10:50:25,  4.19s/it]
  7%|▋         | 680/10000 [48:47<10:44:18,  4.15s/it]
                                                      

  7%|▋         | 680/10000 [48:47<10:44:18,  4.15s/it]
  7%|▋         | 681/10000 [48:51<10:30:05,  4.06s/it]
  7%|▋         | 682/10000 [48:55<10:47:25,  4.17s/it]
  7%|▋         | 683/10000 [48:59<10:45:44,  4.16s/it]
  7%|▋         | 684/10000 [49:04<10:43:42,  4.15s/it]
  7%|▋         | 685/10000 [49:08<10:41:23,  4.13s/it]
  7%|▋         | 686/10000 [49:12<10:54:48,  4.22s/it]
  7%|▋         | 687/10000 [49:16<10:34:33,  4.09s/it]
  7%|▋         | 688/10000 [49:20<10:34:10,  4.09s/it]
  7%|▋         | 689/10000 [49:24<10:48:31,  4.18s/it]
  7%|▋         | 690/10000 [49:28<10:43:46,  4.15s/it]
  7%|▋         | 691/10000 [49:33<10:43:11,  4.15s/it]
  7%|▋         | 692/10000 [49:37<10:38:24,  4.12s/it]
  7%|▋         | 693/10000 [49:41<10:43:37,  4.15s/it]
  7%|▋         | 694/10000 [49:45<10:47:20,  4.17s/it]
  7%|▋         | 695/10000 [49:49<10:42:37,  4.14s/it]
  7%|▋         | 696/10000 [49:53<10:53:11,  4.21s/it]
  7%|▋         | 697/10000 [49:58<10:46:54,  4.17s/it]
  7%|▋         | 698/10000 [50:02<10:42:42,  4.15s/it]
  7%|▋         | 699/10000 [50:06<10:30:50,  4.07s/it]
  7%|▋         | 700/10000 [50:10<10:57:01,  4.24s/it]
                                                      

  7%|▋         | 700/10000 [50:10<10:57:01,  4.24s/it]
  7%|▋         | 701/10000 [50:14<10:50:19,  4.20s/it]
  7%|▋         | 702/10000 [50:18<10:45:36,  4.17s/it]
  7%|▋         | 703/10000 [50:23<10:59:15,  4.25s/it]
  7%|▋         | 704/10000 [50:27<10:54:18,  4.22s/it]
  7%|▋         | 705/10000 [50:31<11:05:10,  4.29s/it]
  7%|▋         | 706/10000 [50:36<10:57:50,  4.25s/it]
  7%|▋         | 707/10000 [50:40<11:02:39,  4.28s/it]
  7%|▋         | 708/10000 [50:44<10:54:22,  4.23s/it]
  7%|▋         | 709/10000 [50:48<10:40:00,  4.13s/it]
  7%|▋         | 710/10000 [50:52<10:55:05,  4.23s/it]
  7%|▋         | 711/10000 [50:57<11:13:43,  4.35s/it]
  7%|▋         | 712/10000 [51:01<10:53:22,  4.22s/it]
  7%|▋         | 713/10000 [51:05<10:36:44,  4.11s/it]
  7%|▋         | 714/10000 [51:09<10:48:29,  4.19s/it]
  7%|▋         | 715/10000 [51:14<10:58:43,  4.26s/it]
  7%|▋         | 716/10000 [51:18<10:56:10,  4.24s/it]
  7%|▋         | 717/10000 [51:22<10:53:50,  4.23s/it]
  7%|▋         | 718/10000 [51:27<11:06:19,  4.31s/it]
  7%|▋         | 719/10000 [51:30<10:48:52,  4.19s/it]
  7%|▋         | 720/10000 [51:35<10:43:52,  4.16s/it]
                                                      

  7%|▋         | 720/10000 [51:35<10:43:52,  4.16s/it]
  7%|▋         | 721/10000 [51:39<10:55:53,  4.24s/it]
  7%|▋         | 722/10000 [51:43<10:50:35,  4.21s/it]
  7%|▋         | 723/10000 [51:47<10:59:09,  4.26s/it]
  7%|▋         | 724/10000 [51:51<10:43:41,  4.16s/it]
  7%|▋         | 725/10000 [51:56<11:00:49,  4.27s/it]
  7%|▋         | 726/10000 [52:00<11:01:59,  4.28s/it]
  7%|▋         | 727/10000 [52:04<11:00:46,  4.28s/it]
  7%|▋         | 728/10000 [52:09<11:01:23,  4.28s/it]
  7%|▋         | 729/10000 [52:13<11:10:42,  4.34s/it]
  7%|▋         | 730/10000 [52:17<11:04:17,  4.30s/it]
  7%|▋         | 731/10000 [52:22<10:53:34,  4.23s/it]
  7%|▋         | 732/10000 [52:26<11:22:40,  4.42s/it]
  7%|▋         | 733/10000 [52:31<11:14:33,  4.37s/it]
  7%|▋         | 734/10000 [52:35<11:07:13,  4.32s/it]
  7%|▋         | 735/10000 [52:39<11:15:23,  4.37s/it]
  7%|▋         | 736/10000 [52:44<11:17:04,  4.39s/it]
  7%|▋         | 737/10000 [52:48<10:55:31,  4.25s/it]
  7%|▋         | 738/10000 [52:52<10:50:15,  4.21s/it]
  7%|▋         | 739/10000 [52:56<10:53:08,  4.23s/it]
  7%|▋         | 740/10000 [53:00<10:57:59,  4.26s/it]
                                                      
{'loss': '-0.02045', 'grad_norm': '6.804', 'learning_rate': '4.721e-06', 'num_tokens': '1.659e+05', 'completions/mean_length': '4.175', 'completions/min_length': '3.35', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.175', 'completions/min_terminated_length': '3.35', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4616', 'rewards/reward_semantic/std': '0.1001', 'rewards/reward_length/mean': '-0.05796', 'rewards/reward_length/std': '0.01356', 'reward': '1.136', 'reward_std': '0.03014', 'frac_reward_zero_std': '0.3', 'kl': '0.2313', 'entropy': '0.4685', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.216', 'epoch': '0.09186'}
{'loss': '0.06506', 'grad_norm': '4.22', 'learning_rate': '4.711e-06', 'num_tokens': '1.719e+05', 'completions/mean_length': '4.45', 'completions/min_length': '3.75', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.45', 'completions/min_terminated_length': '3.75', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4109', 'rewards/reward_semantic/std': '0.05746', 'rewards/reward_length/mean': '-0.04395', 'rewards/reward_length/std': '0.00585', 'reward': '1.121', 'reward_std': '0.01732', 'frac_reward_zero_std': '0.5', 'kl': '0.16', 'entropy': '0.5368', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.138', 'epoch': '0.09514'}
{'loss': '0.003518', 'grad_norm': '12.4', 'learning_rate': '4.701e-06', 'num_tokens': '1.778e+05', 'completions/mean_length': '4.469', 'completions/min_length': '3.3', 'completions/max_length': '6.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '3.3', 'completions/max_terminated_length': '6.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2251', 'rewards/reward_semantic/std': '0.03863', 'rewards/reward_length/mean': '-0.05112', 'rewards/reward_length/std': '0.01265', 'reward': '1.065', 'reward_std': '0.01194', 'frac_reward_zero_std': '0.3', 'kl': '0.5917', 'entropy': '0.5939', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.217', 'epoch': '0.09843'}
{'loss': '0.04598', 'grad_norm': '4.166e-09', 'learning_rate': '4.691e-06', 'num_tokens': '1.84e+05', 'completions/mean_length': '4.819', 'completions/min_length': '4', 'completions/max_length': '5.4', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.819', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4255', 'rewards/reward_semantic/std': '0.06424', 'rewards/reward_length/mean': '-0.04832', 'rewards/reward_length/std': '0.007672', 'reward': '1.125', 'reward_std': '0.01931', 'frac_reward_zero_std': '0.5', 'kl': '0.3023', 'entropy': '0.2533', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.133', 'epoch': '0.1017'}
{'loss': '0.01974', 'grad_norm': '4.886e-05', 'learning_rate': '4.681e-06', 'num_tokens': '1.905e+05', 'completions/mean_length': '5.269', 'completions/min_length': '4.5', 'completions/max_length': '6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.269', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3849', 'rewards/reward_semantic/std': '0.02717', 'rewards/reward_length/mean': '-0.03733', 'rewards/reward_length/std': '0.007423', 'reward': '1.114', 'reward_std': '0.00839', 'frac_reward_zero_std': '0.55', 'kl': '0.8051', 'entropy': '0.2076', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.181', 'epoch': '0.105'}
{'loss': '0.03865', 'grad_norm': '1.287', 'learning_rate': '4.671e-06', 'num_tokens': '1.965e+05', 'completions/mean_length': '4.75', 'completions/min_length': '4.3', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.75', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3838', 'rewards/reward_semantic/std': '0.09965', 'rewards/reward_length/mean': '-0.03083', 'rewards/reward_length/std': '0.007267', 'reward': '1.114', 'reward_std': '0.02979', 'frac_reward_zero_std': '0.6', 'kl': '0.5665', 'entropy': '0.3083', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.087', 'epoch': '0.1083'}
{'loss': '0.0444', 'grad_norm': '0.04157', 'learning_rate': '4.661e-06', 'num_tokens': '2.029e+05', 'completions/mean_length': '5.306', 'completions/min_length': '4.6', 'completions/max_length': '5.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.306', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.25', 'rewards/reward_semantic/std': '0.03536', 'rewards/reward_length/mean': '-0.02852', 'rewards/reward_length/std': '0.003811', 'reward': '1.074', 'reward_std': '0.01062', 'frac_reward_zero_std': '0.7', 'kl': '0.8595', 'entropy': '0.2317', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.154', 'epoch': '0.1115'}
{'loss': '0.01157', 'grad_norm': '24.81', 'learning_rate': '4.651e-06', 'num_tokens': '2.088e+05', 'completions/mean_length': '4.975', 'completions/min_length': '4.35', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.975', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3677', 'rewards/reward_semantic/std': '0.06539', 'rewards/reward_length/mean': '-0.0311', 'rewards/reward_length/std': '0.004359', 'reward': '1.109', 'reward_std': '0.01965', 'frac_reward_zero_std': '0.6', 'kl': '0.5824', 'entropy': '0.1198', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.14', 'epoch': '0.1148'}
{'loss': '0.06939', 'grad_norm': '8.865', 'learning_rate': '4.641e-06', 'num_tokens': '2.145e+05', 'completions/mean_length': '5.081', 'completions/min_length': '3.8', 'completions/max_length': '6.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.081', 'completions/min_terminated_length': '3.8', 'completions/max_terminated_length': '6.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3127', 'rewards/reward_semantic/std': '0.1418', 'rewards/reward_length/mean': '-0.04534', 'rewards/reward_length/std': '0.009209', 'reward': '1.092', 'reward_std': '0.04269', 'frac_reward_zero_std': '0.35', 'kl': '0.7239', 'entropy': '0.2916', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.199', 'epoch': '0.1181'}

  7%|▋         | 740/10000 [53:00<10:57:59,  4.26s/it]
  7%|▋         | 741/10000 [53:04<10:48:25,  4.20s/it]
  7%|▋         | 742/10000 [53:09<10:59:49,  4.28s/it]
  7%|▋         | 743/10000 [53:13<10:56:09,  4.25s/it]
  7%|▋         | 744/10000 [53:17<10:55:54,  4.25s/it]
  7%|▋         | 745/10000 [53:22<10:54:32,  4.24s/it]
  7%|▋         | 746/10000 [53:26<11:08:03,  4.33s/it]
  7%|▋         | 747/10000 [53:30<10:57:18,  4.26s/it]
  7%|▋         | 748/10000 [53:34<10:53:49,  4.24s/it]
  7%|▋         | 749/10000 [53:39<10:57:25,  4.26s/it]
  8%|▊         | 750/10000 [53:43<10:52:30,  4.23s/it]
  8%|▊         | 751/10000 [53:47<10:49:08,  4.21s/it]
  8%|▊         | 752/10000 [53:51<10:45:40,  4.19s/it]
  8%|▊         | 753/10000 [53:56<10:56:21,  4.26s/it]
  8%|▊         | 754/10000 [54:00<10:50:11,  4.22s/it]
  8%|▊         | 755/10000 [54:04<10:57:06,  4.26s/it]
  8%|▊         | 756/10000 [54:09<11:04:28,  4.31s/it]
  8%|▊         | 757/10000 [54:13<11:01:55,  4.30s/it]
  8%|▊         | 758/10000 [54:17<11:02:04,  4.30s/it]
  8%|▊         | 759/10000 [54:21<10:59:37,  4.28s/it]
  8%|▊         | 760/10000 [54:26<11:06:44,  4.33s/it]
                                                      

  8%|▊         | 760/10000 [54:26<11:06:44,  4.33s/it]
  8%|▊         | 761/10000 [54:30<10:57:27,  4.27s/it]
  8%|▊         | 762/10000 [54:34<10:51:27,  4.23s/it]
  8%|▊         | 763/10000 [54:39<11:01:52,  4.30s/it]
  8%|▊         | 764/10000 [54:43<10:50:53,  4.23s/it]
  8%|▊         | 765/10000 [54:47<10:53:50,  4.25s/it]
  8%|▊         | 766/10000 [54:51<11:00:39,  4.29s/it]
  8%|▊         | 767/10000 [54:56<11:07:54,  4.34s/it]
  8%|▊         | 768/10000 [55:00<10:59:20,  4.29s/it]
  8%|▊         | 769/10000 [55:04<11:03:52,  4.32s/it]
  8%|▊         | 770/10000 [55:09<11:10:32,  4.36s/it]
  8%|▊         | 771/10000 [55:13<10:59:43,  4.29s/it]
  8%|▊         | 772/10000 [55:17<11:03:49,  4.32s/it]
  8%|▊         | 773/10000 [55:22<11:03:15,  4.31s/it]
  8%|▊         | 774/10000 [55:26<11:10:02,  4.36s/it]
  8%|▊         | 775/10000 [55:30<11:01:18,  4.30s/it]
  8%|▊         | 776/10000 [55:34<10:43:38,  4.19s/it]
  8%|▊         | 777/10000 [55:38<10:42:28,  4.18s/it]
  8%|▊         | 778/10000 [55:43<10:54:54,  4.26s/it]
  8%|▊         | 779/10000 [55:47<10:57:20,  4.28s/it]
  8%|▊         | 780/10000 [55:51<10:52:05,  4.24s/it]
                                                      

  8%|▊         | 780/10000 [55:51<10:52:05,  4.24s/it]
  8%|▊         | 781/10000 [55:56<11:01:17,  4.30s/it]
  8%|▊         | 782/10000 [56:00<10:55:21,  4.27s/it]
  8%|▊         | 783/10000 [56:04<10:50:07,  4.23s/it]
  8%|▊         | 784/10000 [56:08<10:47:18,  4.21s/it]
  8%|▊         | 785/10000 [56:13<10:58:49,  4.29s/it]
  8%|▊         | 786/10000 [56:17<10:53:12,  4.25s/it]
  8%|▊         | 787/10000 [56:21<10:47:15,  4.22s/it]
  8%|▊         | 788/10000 [56:25<10:59:03,  4.29s/it]
  8%|▊         | 789/10000 [56:30<10:57:25,  4.28s/it]
  8%|▊         | 790/10000 [56:34<10:51:31,  4.24s/it]
  8%|▊         | 791/10000 [56:38<10:47:34,  4.22s/it]
  8%|▊         | 792/10000 [56:42<10:58:23,  4.29s/it]
  8%|▊         | 793/10000 [56:47<10:58:47,  4.29s/it]
  8%|▊         | 794/10000 [56:51<10:51:37,  4.25s/it]
  8%|▊         | 795/10000 [56:55<10:59:23,  4.30s/it]
  8%|▊         | 796/10000 [56:59<10:51:41,  4.25s/it]
  8%|▊         | 797/10000 [57:04<10:47:01,  4.22s/it]
  8%|▊         | 798/10000 [57:08<10:43:39,  4.20s/it]
  8%|▊         | 799/10000 [57:12<10:54:31,  4.27s/it]
  8%|▊         | 800/10000 [57:16<10:48:06,  4.23s/it]
                                                      

  8%|▊         | 800/10000 [57:16<10:48:06,  4.23s/it]
  8%|▊         | 801/10000 [57:20<10:43:52,  4.20s/it]
  8%|▊         | 802/10000 [57:25<11:07:16,  4.35s/it]
  8%|▊         | 803/10000 [57:29<10:57:41,  4.29s/it]
  8%|▊         | 804/10000 [57:33<10:50:52,  4.25s/it]
  8%|▊         | 805/10000 [57:38<10:47:08,  4.22s/it]
  8%|▊         | 806/10000 [57:42<11:06:00,  4.35s/it]
  8%|▊         | 807/10000 [57:46<10:56:58,  4.29s/it]
  8%|▊         | 808/10000 [57:50<10:49:46,  4.24s/it]
  8%|▊         | 809/10000 [57:55<10:45:35,  4.21s/it]
  8%|▊         | 810/10000 [57:59<10:55:18,  4.28s/it]
  8%|▊         | 811/10000 [58:03<11:00:22,  4.31s/it]
  8%|▊         | 812/10000 [58:08<11:04:55,  4.34s/it]
  8%|▊         | 813/10000 [58:12<11:09:26,  4.37s/it]
  8%|▊         | 814/10000 [58:16<10:58:21,  4.30s/it]
  8%|▊         | 815/10000 [58:21<10:52:29,  4.26s/it]
  8%|▊         | 816/10000 [58:25<11:03:01,  4.33s/it]
  8%|▊         | 817/10000 [58:29<10:55:09,  4.28s/it]
  8%|▊         | 818/10000 [58:33<10:49:21,  4.24s/it]
  8%|▊         | 819/10000 [58:38<10:44:50,  4.21s/it]
  8%|▊         | 820/10000 [58:42<10:52:33,  4.27s/it]
                                                      

  8%|▊         | 820/10000 [58:42<10:52:33,  4.27s/it]
  8%|▊         | 821/10000 [58:46<10:46:44,  4.23s/it]
  8%|▊         | 822/10000 [58:50<10:43:42,  4.21s/it]
  8%|▊         | 823/10000 [58:55<10:54:49,  4.28s/it]
  8%|▊         | 824/10000 [58:59<10:46:53,  4.23s/it]
  8%|▊         | 825/10000 [59:03<10:42:06,  4.20s/it]
  8%|▊         | 826/10000 [59:07<10:40:35,  4.19s/it]
  8%|▊         | 827/10000 [59:12<10:51:14,  4.26s/it]
  8%|▊         | 828/10000 [59:16<10:43:01,  4.21s/it]
  8%|▊         | 829/10000 [59:20<10:39:00,  4.18s/it]
  8%|▊         | 830/10000 [59:24<10:35:54,  4.16s/it]
  8%|▊         | 831/10000 [59:28<10:46:53,  4.23s/it]
  8%|▊         | 832/10000 [59:32<10:41:44,  4.20s/it]
  8%|▊         | 833/10000 [59:37<10:38:01,  4.18s/it]
  8%|▊         | 834/10000 [59:41<11:11:21,  4.39s/it]
  8%|▊         | 835/10000 [59:46<11:17:19,  4.43s/it]
  8%|▊         | 836/10000 [59:50<11:00:27,  4.32s/it]
  8%|▊         | 837/10000 [59:54<10:53:25,  4.28s/it]
  8%|▊         | 838/10000 [59:59<11:06:01,  4.36s/it]
  8%|▊         | 839/10000 [1:00:03<10:55:51,  4.30s/it]
  8%|▊         | 840/10000 [1:00:07<10:48:49,  4.25s/it]
                                                        

  8%|▊         | 840/10000 [1:00:07<10:48:49,  4.25s/it]
  8%|▊         | 841/10000 [1:00:11<10:57:52,  4.31s/it]
  8%|▊         | 842/10000 [1:00:16<10:50:03,  4.26s/it]
  8%|▊         | 843/10000 [1:00:20<10:46:16,  4.23s/it]
  8%|▊         | 844/10000 [1:00:24<10:43:04,  4.21s/it]
  8%|▊         | 845/10000 [1:00:28<10:52:48,  4.28s/it]
  8%|▊         | 846/10000 [1:00:33<10:47:00,  4.24s/it]
  8%|▊         | 847/10000 [1:00:37<10:42:31,  4.21s/it]
  8%|▊         | 848/10000 [1:00:41<10:53:40,  4.29s/it]
  8%|▊         | 849/10000 [1:00:45<10:46:37,  4.24s/it]
  8%|▊         | 850/10000 [1:00:49<10:42:53,  4.22s/it]
  9%|▊         | 851/10000 [1:00:54<10:37:37,  4.18s/it]
  9%|▊         | 852/10000 [1:00:58<10:47:52,  4.25s/it]
  9%|▊         | 853/10000 [1:01:02<10:43:05,  4.22s/it]
  9%|▊         | 854/10000 [1:01:06<10:39:36,  4.20s/it]
  9%|▊         | 855/10000 [1:01:11<10:49:46,  4.26s/it]
  9%|▊         | 856/10000 [1:01:15<10:53:11,  4.29s/it]
  9%|▊         | 857/10000 [1:01:19<10:50:37,  4.27s/it]
  9%|▊         | 858/10000 [1:01:23<10:45:30,  4.24s/it]
  9%|▊         | 859/10000 [1:01:28<10:55:55,  4.31s/it]
  9%|▊         | 860/10000 [1:01:32<10:49:01,  4.26s/it]
                                                        

  9%|▊         | 860/10000 [1:01:32<10:49:01,  4.26s/it]
  9%|▊         | 861/10000 [1:01:36<10:43:08,  4.22s/it]
  9%|▊         | 862/10000 [1:01:41<10:52:39,  4.29s/it]
  9%|▊         | 863/10000 [1:01:45<10:48:10,  4.26s/it]
  9%|▊         | 864/10000 [1:01:49<10:46:26,  4.25s/it]
  9%|▊         | 865/10000 [1:01:53<10:40:22,  4.21s/it]
  9%|▊         | 866/10000 [1:01:58<10:54:27,  4.30s/it]
  9%|▊         | 867/10000 [1:02:02<10:55:14,  4.30s/it]
  9%|▊         | 868/10000 [1:02:06<11:03:05,  4.36s/it]
  9%|▊         | 869/10000 [1:02:11<11:11:11,  4.41s/it]
  9%|▊         | 870/10000 [1:02:15<11:09:30,  4.40s/it]
  9%|▊         | 871/10000 [1:02:20<11:06:16,  4.38s/it]
  9%|▊         | 872/10000 [1:02:24<10:54:19,  4.30s/it]
  9%|▊         | 873/10000 [1:02:28<10:58:52,  4.33s/it]
  9%|▊         | 874/10000 [1:02:32<10:43:22,  4.23s/it]
  9%|▉         | 875/10000 [1:02:36<10:38:50,  4.20s/it]
  9%|▉         | 876/10000 [1:02:40<10:39:22,  4.20s/it]
  9%|▉         | 877/10000 [1:02:45<10:44:37,  4.24s/it]
  9%|▉         | 878/10000 [1:02:49<10:29:47,  4.14s/it]
  9%|▉         | 879/10000 [1:02:53<10:28:38,  4.14s/it]
  9%|▉         | 880/10000 [1:02:57<10:40:31,  4.21s/it]
                                                        

  9%|▉         | 880/10000 [1:02:57<10:40:31,  4.21s/it]
  9%|▉         | 881/10000 [1:03:02<10:48:44,  4.27s/it]
  9%|▉         | 882/10000 [1:03:06<10:41:47,  4.22s/it]
  9%|▉         | 883/10000 [1:03:10<10:59:44,  4.34s/it]
  9%|▉         | 884/10000 [1:03:14<10:40:09,  4.21s/it]
  9%|▉         | 885/10000 [1:03:19<10:48:22,  4.27s/it]
  9%|▉         | 886/10000 [1:03:23<10:46:36,  4.26s/it]
  9%|▉         | 887/10000 [1:03:27<10:47:05,  4.26s/it]
  9%|▉         | 888/10000 [1:03:32<10:59:39,  4.34s/it]
  9%|▉         | 889/10000 [1:03:36<11:17:33,  4.46s/it]
  9%|▉         | 890/10000 [1:03:41<11:24:45,  4.51s/it]
  9%|▉         | 891/10000 [1:03:45<11:09:33,  4.41s/it]
  9%|▉         | 892/10000 [1:03:49<10:46:44,  4.26s/it]
  9%|▉         | 893/10000 [1:03:53<10:44:26,  4.25s/it]
  9%|▉         | 894/10000 [1:03:58<10:53:26,  4.31s/it]
  9%|▉         | 895/10000 [1:04:02<10:48:27,  4.27s/it]
  9%|▉         | 896/10000 [1:04:06<10:44:42,  4.25s/it]
  9%|▉         | 897/10000 [1:04:11<10:47:26,  4.27s/it]
  9%|▉         | 898/10000 [1:04:16<11:22:47,  4.50s/it]
  9%|▉         | 899/10000 [1:04:20<11:05:20,  4.39s/it]
  9%|▉         | 900/10000 [1:04:24<11:05:55,  4.39s/it]
                                                        

  9%|▉         | 900/10000 [1:04:24<11:05:55,  4.39s/it]
  9%|▉         | 901/10000 [1:04:29<11:21:49,  4.50s/it]
  9%|▉         | 902/10000 [1:04:33<11:10:28,  4.42s/it]
  9%|▉         | 903/10000 [1:04:37<10:54:07,  4.31s/it]
  9%|▉         | 904/10000 [1:04:42<11:00:52,  4.36s/it]
  9%|▉         | 905/10000 [1:04:46<10:52:09,  4.30s/it]
  9%|▉         | 906/10000 [1:04:50<10:56:58,  4.33s/it]
  9%|▉         | 907/10000 [1:04:54<10:47:50,  4.27s/it]
  9%|▉         | 908/10000 [1:04:59<10:54:22,  4.32s/it]
  9%|▉         | 909/10000 [1:05:03<10:39:36,  4.22s/it]
  9%|▉         | 910/10000 [1:05:07<10:44:41,  4.26s/it]
  9%|▉         | 911/10000 [1:05:12<11:00:04,  4.36s/it]
  9%|▉         | 912/10000 [1:05:16<10:46:01,  4.27s/it]
  9%|▉         | 913/10000 [1:05:20<10:52:35,  4.31s/it]
  9%|▉         | 914/10000 [1:05:24<10:40:23,  4.23s/it]
  9%|▉         | 915/10000 [1:05:29<10:47:50,  4.28s/it]
  9%|▉         | 916/10000 [1:05:33<10:47:13,  4.27s/it]
  9%|▉         | 917/10000 [1:05:37<10:30:19,  4.16s/it]
  9%|▉         | 918/10000 [1:05:41<10:56:07,  4.33s/it]
  9%|▉         | 919/10000 [1:05:46<10:48:10,  4.28s/it]
  9%|▉         | 920/10000 [1:05:50<10:32:21,  4.18s/it]
                                                        
{'loss': '0.02833', 'grad_norm': '15.3', 'learning_rate': '4.631e-06', 'num_tokens': '2.206e+05', 'completions/mean_length': '4.812', 'completions/min_length': '4.05', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.812', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4289', 'rewards/reward_semantic/std': '0.1145', 'rewards/reward_length/mean': '-0.04504', 'rewards/reward_length/std': '0.00661', 'reward': '1.126', 'reward_std': '0.03431', 'frac_reward_zero_std': '0.4', 'kl': '0.6375', 'entropy': '0.2863', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.276', 'epoch': '0.1214'}
{'loss': '0.07976', 'grad_norm': '0.003008', 'learning_rate': '4.621e-06', 'num_tokens': '2.263e+05', 'completions/mean_length': '5.169', 'completions/min_length': '4.8', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.169', 'completions/min_terminated_length': '4.8', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3208', 'rewards/reward_semantic/std': '0.02315', 'rewards/reward_length/mean': '-0.02551', 'rewards/reward_length/std': '0.001799', 'reward': '1.095', 'reward_std': '0.006957', 'frac_reward_zero_std': '0.6', 'kl': '1.545', 'entropy': '0.1076', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.249', 'epoch': '0.1247'}
{'loss': '0.08253', 'grad_norm': '0.001607', 'learning_rate': '4.611e-06', 'num_tokens': '2.318e+05', 'completions/mean_length': '4.931', 'completions/min_length': '4.5', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.931', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1391', 'rewards/reward_semantic/std': '0.03977', 'rewards/reward_length/mean': '-0.02661', 'rewards/reward_length/std': '0.004281', 'reward': '1.04', 'reward_std': '0.01189', 'frac_reward_zero_std': '0.7', 'kl': '2.151', 'entropy': '0.1121', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.25', 'epoch': '0.128'}
{'loss': '0.1423', 'grad_norm': '0.01201', 'learning_rate': '4.601e-06', 'num_tokens': '2.379e+05', 'completions/mean_length': '4.769', 'completions/min_length': '4.65', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.769', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.175', 'rewards/reward_semantic/std': '0.04356', 'rewards/reward_length/mean': '-0.01697', 'rewards/reward_length/std': '0.001743', 'reward': '1.052', 'reward_std': '0.01312', 'frac_reward_zero_std': '0.85', 'kl': '2.518', 'entropy': '0.07226', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.233', 'epoch': '0.1312'}
{'loss': '0.1234', 'grad_norm': '18.89', 'learning_rate': '4.591e-06', 'num_tokens': '2.441e+05', 'completions/mean_length': '4.713', 'completions/min_length': '4.35', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.713', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1471', 'rewards/reward_semantic/std': '0.02367', 'rewards/reward_length/mean': '-0.02598', 'rewards/reward_length/std': '0.006262', 'reward': '1.043', 'reward_std': '0.007229', 'frac_reward_zero_std': '0.65', 'kl': '2.782', 'entropy': '0.1545', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.265', 'epoch': '0.1345'}
{'loss': '0.1035', 'grad_norm': '0.032', 'learning_rate': '4.581e-06', 'num_tokens': '2.502e+05', 'completions/mean_length': '5.219', 'completions/min_length': '4.75', 'completions/max_length': '5.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.219', 'completions/min_terminated_length': '4.75', 'completions/max_terminated_length': '5.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1854', 'rewards/reward_semantic/std': '0.07207', 'rewards/reward_length/mean': '-0.02683', 'rewards/reward_length/std': '0.00808', 'reward': '1.054', 'reward_std': '0.02157', 'frac_reward_zero_std': '0.7', 'kl': '2.708', 'entropy': '0.1375', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.233', 'epoch': '0.1378'}
{'loss': '0.1069', 'grad_norm': '0.001654', 'learning_rate': '4.571e-06', 'num_tokens': '2.563e+05', 'completions/mean_length': '4.894', 'completions/min_length': '4.55', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.894', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3366', 'rewards/reward_semantic/std': '0.05992', 'rewards/reward_length/mean': '-0.01992', 'rewards/reward_length/std': '0.006409', 'reward': '1.1', 'reward_std': '0.01771', 'frac_reward_zero_std': '0.8', 'kl': '3.046', 'entropy': '0.1199', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.229', 'epoch': '0.1411'}
{'loss': '0.09207', 'grad_norm': '0.005598', 'learning_rate': '4.561e-06', 'num_tokens': '2.625e+05', 'completions/mean_length': '4.594', 'completions/min_length': '4.15', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.594', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3684', 'rewards/reward_semantic/std': '0.07189', 'rewards/reward_length/mean': '-0.03542', 'rewards/reward_length/std': '0.01001', 'reward': '1.109', 'reward_std': '0.0216', 'frac_reward_zero_std': '0.5', 'kl': '1.586', 'entropy': '0.2552', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.243', 'epoch': '0.1444'}
{'loss': '0.1127', 'grad_norm': '12.64', 'learning_rate': '4.551e-06', 'num_tokens': '2.684e+05', 'completions/mean_length': '4.263', 'completions/min_length': '3.1', 'completions/max_length': '6.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.263', 'completions/min_terminated_length': '3.1', 'completions/max_terminated_length': '6.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3548', 'rewards/reward_semantic/std': '0.06994', 'rewards/reward_length/mean': '-0.04905', 'rewards/reward_length/std': '0.01453', 'reward': '1.104', 'reward_std': '0.02128', 'frac_reward_zero_std': '0.2', 'kl': '0.7776', 'entropy': '0.7876', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.323', 'epoch': '0.1476'}

  9%|▉         | 920/10000 [1:05:50<10:32:21,  4.18s/it]
  9%|▉         | 921/10000 [1:05:54<10:24:17,  4.13s/it]
  9%|▉         | 922/10000 [1:05:58<10:29:14,  4.16s/it]
  9%|▉         | 923/10000 [1:06:02<10:28:27,  4.15s/it]
  9%|▉         | 924/10000 [1:06:07<10:51:06,  4.30s/it]
  9%|▉         | 925/10000 [1:06:11<10:46:54,  4.28s/it]
  9%|▉         | 926/10000 [1:06:15<10:30:57,  4.17s/it]
  9%|▉         | 927/10000 [1:06:19<10:31:05,  4.17s/it]
  9%|▉         | 928/10000 [1:06:23<10:42:59,  4.25s/it]
  9%|▉         | 929/10000 [1:06:28<10:38:58,  4.23s/it]
  9%|▉         | 930/10000 [1:06:32<10:35:14,  4.20s/it]
  9%|▉         | 931/10000 [1:06:36<10:41:27,  4.24s/it]
  9%|▉         | 932/10000 [1:06:41<11:05:17,  4.40s/it]
  9%|▉         | 933/10000 [1:06:45<10:43:48,  4.26s/it]
  9%|▉         | 934/10000 [1:06:49<10:37:14,  4.22s/it]
  9%|▉         | 935/10000 [1:06:53<10:53:10,  4.32s/it]
  9%|▉         | 936/10000 [1:06:58<11:07:21,  4.42s/it]
  9%|▉         | 937/10000 [1:07:02<11:04:55,  4.40s/it]
  9%|▉         | 938/10000 [1:07:06<10:49:12,  4.30s/it]
  9%|▉         | 939/10000 [1:07:11<11:04:53,  4.40s/it]
  9%|▉         | 940/10000 [1:07:15<10:44:05,  4.27s/it]
                                                        

  9%|▉         | 940/10000 [1:07:15<10:44:05,  4.27s/it]
  9%|▉         | 941/10000 [1:07:19<10:48:28,  4.29s/it]
  9%|▉         | 942/10000 [1:07:24<11:02:25,  4.39s/it]
  9%|▉         | 943/10000 [1:07:28<10:41:27,  4.25s/it]
  9%|▉         | 944/10000 [1:07:32<10:37:35,  4.22s/it]
  9%|▉         | 945/10000 [1:07:36<10:43:10,  4.26s/it]
  9%|▉         | 946/10000 [1:07:41<11:02:54,  4.39s/it]
  9%|▉         | 947/10000 [1:07:45<10:56:08,  4.35s/it]
  9%|▉         | 948/10000 [1:07:50<10:59:08,  4.37s/it]
  9%|▉         | 949/10000 [1:07:54<10:53:02,  4.33s/it]
 10%|▉         | 950/10000 [1:07:58<10:43:09,  4.26s/it]
 10%|▉         | 951/10000 [1:08:02<10:36:25,  4.22s/it]
 10%|▉         | 952/10000 [1:08:06<10:34:17,  4.21s/it]
 10%|▉         | 953/10000 [1:08:11<10:46:57,  4.29s/it]
 10%|▉         | 954/10000 [1:08:15<10:40:57,  4.25s/it]
 10%|▉         | 955/10000 [1:08:19<10:39:15,  4.24s/it]
 10%|▉         | 956/10000 [1:08:23<10:29:33,  4.18s/it]
 10%|▉         | 957/10000 [1:08:28<10:37:15,  4.23s/it]
 10%|▉         | 958/10000 [1:08:32<10:19:06,  4.11s/it]
 10%|▉         | 959/10000 [1:08:36<10:16:52,  4.09s/it]
 10%|▉         | 960/10000 [1:08:40<10:29:37,  4.18s/it]
                                                        

 10%|▉         | 960/10000 [1:08:40<10:29:37,  4.18s/it]
 10%|▉         | 961/10000 [1:08:44<10:23:34,  4.14s/it]
 10%|▉         | 962/10000 [1:08:48<10:20:32,  4.12s/it]
 10%|▉         | 963/10000 [1:08:52<10:17:02,  4.10s/it]
 10%|▉         | 964/10000 [1:08:57<10:35:51,  4.22s/it]
 10%|▉         | 965/10000 [1:09:01<10:32:03,  4.20s/it]
 10%|▉         | 966/10000 [1:09:05<10:32:47,  4.20s/it]
 10%|▉         | 967/10000 [1:09:09<10:29:29,  4.18s/it]
 10%|▉         | 968/10000 [1:09:13<10:19:44,  4.12s/it]
 10%|▉         | 969/10000 [1:09:17<10:19:54,  4.12s/it]
 10%|▉         | 970/10000 [1:09:21<10:17:14,  4.10s/it]
 10%|▉         | 971/10000 [1:09:26<10:37:15,  4.23s/it]
 10%|▉         | 972/10000 [1:09:30<10:28:41,  4.18s/it]
 10%|▉         | 973/10000 [1:09:34<10:17:00,  4.10s/it]
 10%|▉         | 974/10000 [1:09:38<10:27:09,  4.17s/it]
 10%|▉         | 975/10000 [1:09:42<10:29:12,  4.18s/it]
 10%|▉         | 976/10000 [1:09:46<10:20:33,  4.13s/it]
 10%|▉         | 977/10000 [1:09:50<10:15:44,  4.09s/it]
 10%|▉         | 978/10000 [1:09:55<10:40:42,  4.26s/it]
 10%|▉         | 979/10000 [1:09:59<10:36:27,  4.23s/it]
 10%|▉         | 980/10000 [1:10:03<10:26:18,  4.17s/it]
                                                        

 10%|▉         | 980/10000 [1:10:03<10:26:18,  4.17s/it]
 10%|▉         | 981/10000 [1:10:08<10:41:29,  4.27s/it]
 10%|▉         | 982/10000 [1:10:12<10:31:38,  4.20s/it]
 10%|▉         | 983/10000 [1:10:16<10:23:03,  4.15s/it]
 10%|▉         | 984/10000 [1:10:20<10:17:05,  4.11s/it]
 10%|▉         | 985/10000 [1:10:24<10:24:17,  4.15s/it]
 10%|▉         | 986/10000 [1:10:28<10:21:03,  4.13s/it]
 10%|▉         | 987/10000 [1:10:32<10:15:43,  4.10s/it]
 10%|▉         | 988/10000 [1:10:36<10:04:38,  4.03s/it]
 10%|▉         | 989/10000 [1:10:40<10:17:02,  4.11s/it]
 10%|▉         | 990/10000 [1:10:44<10:14:57,  4.10s/it]
 10%|▉         | 991/10000 [1:10:48<10:14:21,  4.09s/it]
 10%|▉         | 992/10000 [1:10:53<10:25:41,  4.17s/it]
 10%|▉         | 993/10000 [1:10:57<10:19:30,  4.13s/it]
 10%|▉         | 994/10000 [1:11:01<10:14:07,  4.09s/it]
 10%|▉         | 995/10000 [1:11:05<10:23:33,  4.15s/it]
 10%|▉         | 996/10000 [1:11:09<10:18:23,  4.12s/it]
 10%|▉         | 997/10000 [1:11:13<10:13:51,  4.09s/it]
 10%|▉         | 998/10000 [1:11:17<10:10:35,  4.07s/it]
 10%|▉         | 999/10000 [1:11:22<10:21:39,  4.14s/it]
 10%|█         | 1000/10000 [1:11:26<10:16:48,  4.11s/it]
                                                         

 10%|█         | 1000/10000 [1:11:26<10:16:48,  4.11s/it]
 10%|█         | 1001/10000 [1:11:31<11:01:45,  4.41s/it]
 10%|█         | 1002/10000 [1:11:35<10:57:30,  4.38s/it]
 10%|█         | 1003/10000 [1:11:39<10:41:35,  4.28s/it]
 10%|█         | 1004/10000 [1:11:43<10:30:15,  4.20s/it]
 10%|█         | 1005/10000 [1:11:47<10:22:51,  4.15s/it]
 10%|█         | 1006/10000 [1:11:51<10:28:39,  4.19s/it]
 10%|█         | 1007/10000 [1:11:55<10:20:36,  4.14s/it]
 10%|█         | 1008/10000 [1:11:59<10:14:56,  4.10s/it]
 10%|█         | 1009/10000 [1:12:03<10:10:39,  4.08s/it]
 10%|█         | 1010/10000 [1:12:08<10:10:33,  4.07s/it]
 10%|█         | 1011/10000 [1:12:12<10:07:19,  4.05s/it]
 10%|█         | 1012/10000 [1:12:16<10:05:48,  4.04s/it]
 10%|█         | 1013/10000 [1:12:20<10:17:56,  4.13s/it]
 10%|█         | 1014/10000 [1:12:24<10:12:43,  4.09s/it]
 10%|█         | 1015/10000 [1:12:28<10:09:26,  4.07s/it]
 10%|█         | 1016/10000 [1:12:32<10:07:01,  4.05s/it]
 10%|█         | 1017/10000 [1:12:36<10:18:07,  4.13s/it]
 10%|█         | 1018/10000 [1:12:40<10:13:44,  4.10s/it]
 10%|█         | 1019/10000 [1:12:44<10:10:03,  4.08s/it]
 10%|█         | 1020/10000 [1:12:49<10:20:56,  4.15s/it]
                                                         

 10%|█         | 1020/10000 [1:12:49<10:20:56,  4.15s/it]
 10%|█         | 1021/10000 [1:12:53<10:15:12,  4.11s/it]
 10%|█         | 1022/10000 [1:12:57<10:11:13,  4.08s/it]
 10%|█         | 1023/10000 [1:13:01<10:08:54,  4.07s/it]
 10%|█         | 1024/10000 [1:13:05<10:19:31,  4.14s/it]
 10%|█         | 1025/10000 [1:13:09<10:14:03,  4.11s/it]
 10%|█         | 1026/10000 [1:13:13<10:07:59,  4.06s/it]
 10%|█         | 1027/10000 [1:13:17<10:18:46,  4.14s/it]
 10%|█         | 1028/10000 [1:13:22<10:24:30,  4.18s/it]
 10%|█         | 1029/10000 [1:13:26<10:16:51,  4.13s/it]
 10%|█         | 1030/10000 [1:13:30<10:24:54,  4.18s/it]
 10%|█         | 1031/10000 [1:13:34<10:17:46,  4.13s/it]
 10%|█         | 1032/10000 [1:13:38<10:12:41,  4.10s/it]
 10%|█         | 1033/10000 [1:13:42<10:08:45,  4.07s/it]
 10%|█         | 1034/10000 [1:13:46<10:19:41,  4.15s/it]
 10%|█         | 1035/10000 [1:13:50<10:13:55,  4.11s/it]
 10%|█         | 1036/10000 [1:13:54<10:14:33,  4.11s/it]
 10%|█         | 1037/10000 [1:13:59<10:23:01,  4.17s/it]
 10%|█         | 1038/10000 [1:14:03<10:16:41,  4.13s/it]
 10%|█         | 1039/10000 [1:14:07<10:12:12,  4.10s/it]
 10%|█         | 1040/10000 [1:14:11<10:08:46,  4.08s/it]
                                                         

 10%|█         | 1040/10000 [1:14:11<10:08:46,  4.08s/it]
 10%|█         | 1041/10000 [1:14:15<10:23:05,  4.17s/it]
 10%|█         | 1042/10000 [1:14:19<10:16:28,  4.13s/it]
 10%|█         | 1043/10000 [1:14:23<10:11:37,  4.10s/it]
 10%|█         | 1044/10000 [1:14:28<10:21:33,  4.16s/it]
 10%|█         | 1045/10000 [1:14:32<10:14:59,  4.12s/it]
 10%|█         | 1046/10000 [1:14:36<10:10:38,  4.09s/it]
 10%|█         | 1047/10000 [1:14:40<10:06:47,  4.07s/it]
 10%|█         | 1048/10000 [1:14:44<10:17:57,  4.14s/it]
 10%|█         | 1049/10000 [1:14:48<10:13:29,  4.11s/it]
 10%|█         | 1050/10000 [1:14:52<10:12:40,  4.11s/it]
 11%|█         | 1051/10000 [1:14:56<10:21:45,  4.17s/it]
 11%|█         | 1052/10000 [1:15:00<10:15:51,  4.13s/it]
 11%|█         | 1053/10000 [1:15:04<10:13:04,  4.11s/it]
 11%|█         | 1054/10000 [1:15:09<10:49:35,  4.36s/it]
 11%|█         | 1055/10000 [1:15:14<10:47:28,  4.34s/it]
 11%|█         | 1056/10000 [1:15:18<10:33:39,  4.25s/it]
 11%|█         | 1057/10000 [1:15:22<10:23:52,  4.19s/it]
 11%|█         | 1058/10000 [1:15:26<10:29:50,  4.23s/it]
 11%|█         | 1059/10000 [1:15:30<10:21:09,  4.17s/it]
 11%|█         | 1060/10000 [1:15:34<10:15:29,  4.13s/it]
                                                         

 11%|█         | 1060/10000 [1:15:34<10:15:29,  4.13s/it]
 11%|█         | 1061/10000 [1:15:38<10:10:54,  4.10s/it]
 11%|█         | 1062/10000 [1:15:43<10:37:01,  4.28s/it]
 11%|█         | 1063/10000 [1:15:47<10:26:34,  4.21s/it]
 11%|█         | 1064/10000 [1:15:51<10:21:32,  4.17s/it]
 11%|█         | 1065/10000 [1:15:55<10:34:02,  4.26s/it]
 11%|█         | 1066/10000 [1:16:00<10:29:54,  4.23s/it]
 11%|█         | 1067/10000 [1:16:04<10:22:10,  4.18s/it]
 11%|█         | 1068/10000 [1:16:08<10:16:04,  4.14s/it]
 11%|█         | 1069/10000 [1:16:12<10:23:40,  4.19s/it]
 11%|█         | 1070/10000 [1:16:16<10:35:14,  4.27s/it]
 11%|█         | 1071/10000 [1:16:21<10:24:09,  4.19s/it]
 11%|█         | 1072/10000 [1:16:25<10:29:08,  4.23s/it]
 11%|█         | 1073/10000 [1:16:29<10:19:56,  4.17s/it]
 11%|█         | 1074/10000 [1:16:33<10:26:58,  4.21s/it]
 11%|█         | 1075/10000 [1:16:37<10:28:01,  4.22s/it]
 11%|█         | 1076/10000 [1:16:42<10:45:16,  4.34s/it]
 11%|█         | 1077/10000 [1:16:46<10:36:29,  4.28s/it]
 11%|█         | 1078/10000 [1:16:50<10:30:27,  4.24s/it]
 11%|█         | 1079/10000 [1:16:55<10:30:07,  4.24s/it]
 11%|█         | 1080/10000 [1:16:59<10:26:43,  4.22s/it]
                                                         

 11%|█         | 1080/10000 [1:16:59<10:26:43,  4.22s/it]
 11%|█         | 1081/10000 [1:17:03<10:22:35,  4.19s/it]
 11%|█         | 1082/10000 [1:17:07<10:19:15,  4.17s/it]
 11%|█         | 1083/10000 [1:17:11<10:34:03,  4.27s/it]
 11%|█         | 1084/10000 [1:17:16<10:28:31,  4.23s/it]
 11%|█         | 1085/10000 [1:17:19<10:14:03,  4.13s/it]
 11%|█         | 1086/10000 [1:17:24<10:17:24,  4.16s/it]
 11%|█         | 1087/10000 [1:17:28<10:10:00,  4.11s/it]
 11%|█         | 1088/10000 [1:17:32<10:10:26,  4.11s/it]
 11%|█         | 1089/10000 [1:17:36<10:20:16,  4.18s/it]
 11%|█         | 1090/10000 [1:17:41<10:31:38,  4.25s/it]
 11%|█         | 1091/10000 [1:17:45<10:27:02,  4.22s/it]
 11%|█         | 1092/10000 [1:17:49<10:14:34,  4.14s/it]
 11%|█         | 1093/10000 [1:17:53<10:36:56,  4.29s/it]
 11%|█         | 1094/10000 [1:17:58<10:46:00,  4.35s/it]
 11%|█         | 1095/10000 [1:18:02<10:44:53,  4.35s/it]
 11%|█         | 1096/10000 [1:18:06<10:42:59,  4.33s/it]
 11%|█         | 1097/10000 [1:18:11<10:48:00,  4.37s/it]
 11%|█         | 1098/10000 [1:18:15<10:27:51,  4.23s/it]
 11%|█         | 1099/10000 [1:18:19<10:24:24,  4.21s/it]
 11%|█         | 1100/10000 [1:18:23<10:34:29,  4.28s/it]
                                                         
{'loss': '0.1154', 'grad_norm': '0.007462', 'learning_rate': '4.541e-06', 'num_tokens': '2.743e+05', 'completions/mean_length': '4.15', 'completions/min_length': '3.45', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.15', 'completions/min_terminated_length': '3.45', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1775', 'rewards/reward_semantic/std': '0.01035', 'rewards/reward_length/mean': '-0.04041', 'rewards/reward_length/std': '0.01252', 'reward': '1.051', 'reward_std': '0.003691', 'frac_reward_zero_std': '0.3', 'kl': '1.258', 'entropy': '0.4647', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.254', 'epoch': '0.1509'}
{'loss': '0.04921', 'grad_norm': '0.002279', 'learning_rate': '4.531e-06', 'num_tokens': '2.801e+05', 'completions/mean_length': '4.044', 'completions/min_length': '3.15', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.044', 'completions/min_terminated_length': '3.15', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3998', 'rewards/reward_semantic/std': '0.08241', 'rewards/reward_length/mean': '-0.04033', 'rewards/reward_length/std': '0.0113', 'reward': '1.118', 'reward_std': '0.0246', 'frac_reward_zero_std': '0.4', 'kl': '1.223', 'entropy': '0.4397', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.254', 'epoch': '0.1542'}
{'loss': '0.06924', 'grad_norm': '16.98', 'learning_rate': '4.521e-06', 'num_tokens': '2.857e+05', 'completions/mean_length': '4.375', 'completions/min_length': '3.8', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.375', 'completions/min_terminated_length': '3.8', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4928', 'rewards/reward_semantic/std': '0.08527', 'rewards/reward_length/mean': '-0.03381', 'rewards/reward_length/std': '0.008382', 'reward': '1.146', 'reward_std': '0.02566', 'frac_reward_zero_std': '0.4', 'kl': '1.282', 'entropy': '0.3127', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.227', 'epoch': '0.1575'}
{'loss': '0.0461', 'grad_norm': '0.002238', 'learning_rate': '4.511e-06', 'num_tokens': '2.915e+05', 'completions/mean_length': '4.763', 'completions/min_length': '4', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.763', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2364', 'rewards/reward_semantic/std': '0.03218', 'rewards/reward_length/mean': '-0.03252', 'rewards/reward_length/std': '0.006001', 'reward': '1.069', 'reward_std': '0.009738', 'frac_reward_zero_std': '0.55', 'kl': '1.753', 'entropy': '0.2208', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.143', 'epoch': '0.1608'}
{'loss': '0.07284', 'grad_norm': '0.08237', 'learning_rate': '4.501e-06', 'num_tokens': '2.97e+05', 'completions/mean_length': '4.625', 'completions/min_length': '4.25', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.625', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2969', 'rewards/reward_semantic/std': '0.03882', 'rewards/reward_length/mean': '-0.02627', 'rewards/reward_length/std': '0.005801', 'reward': '1.088', 'reward_std': '0.01164', 'frac_reward_zero_std': '0.65', 'kl': '2.464', 'entropy': '0.1269', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.101', 'epoch': '0.164'}
{'loss': '0.1046', 'grad_norm': '0.001257', 'learning_rate': '4.491e-06', 'num_tokens': '3.029e+05', 'completions/mean_length': '4.75', 'completions/min_length': '4.7', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.75', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1875', 'rewards/reward_semantic/std': '0.01336', 'rewards/reward_length/mean': '-0.01787', 'rewards/reward_length/std': '0.0009396', 'reward': '1.055', 'reward_std': '0.003962', 'frac_reward_zero_std': '0.95', 'kl': '3.049', 'entropy': '0.03494', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.077', 'epoch': '0.1673'}
{'loss': '0.1426', 'grad_norm': '0.0007985', 'learning_rate': '4.481e-06', 'num_tokens': '3.089e+05', 'completions/mean_length': '4.85', 'completions/min_length': '4.75', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.85', 'completions/min_terminated_length': '4.75', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2516', 'rewards/reward_semantic/std': '0.03553', 'rewards/reward_length/mean': '-0.01775', 'rewards/reward_length/std': '0.002483', 'reward': '1.075', 'reward_std': '0.01061', 'frac_reward_zero_std': '0.9', 'kl': '2.705', 'entropy': '0.04134', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.091', 'epoch': '0.1706'}
{'loss': '0.126', 'grad_norm': '15.24', 'learning_rate': '4.471e-06', 'num_tokens': '3.152e+05', 'completions/mean_length': '5.088', 'completions/min_length': '4.55', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.088', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2368', 'rewards/reward_semantic/std': '0.08032', 'rewards/reward_length/mean': '-0.02529', 'rewards/reward_length/std': '0.004513', 'reward': '1.07', 'reward_std': '0.02397', 'frac_reward_zero_std': '0.75', 'kl': '2.584', 'entropy': '0.07063', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.152', 'epoch': '0.1739'}
{'loss': '0.07044', 'grad_norm': '8.215', 'learning_rate': '4.461e-06', 'num_tokens': '3.213e+05', 'completions/mean_length': '4.8', 'completions/min_length': '4.15', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.8', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2452', 'rewards/reward_semantic/std': '0.04636', 'rewards/reward_length/mean': '-0.03669', 'rewards/reward_length/std': '0.007137', 'reward': '1.072', 'reward_std': '0.0138', 'frac_reward_zero_std': '0.55', 'kl': '1.994', 'entropy': '0.2262', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.209', 'epoch': '0.1772'}

 11%|█         | 1100/10000 [1:18:23<10:34:29,  4.28s/it]
 11%|█         | 1101/10000 [1:18:28<10:29:57,  4.25s/it]
 11%|█         | 1102/10000 [1:18:32<10:25:25,  4.22s/it]
 11%|█         | 1103/10000 [1:18:36<10:22:09,  4.20s/it]
 11%|█         | 1104/10000 [1:18:40<10:30:32,  4.25s/it]
 11%|█         | 1105/10000 [1:18:45<10:34:18,  4.28s/it]
 11%|█         | 1106/10000 [1:18:49<10:25:19,  4.22s/it]
 11%|█         | 1107/10000 [1:18:53<10:35:14,  4.29s/it]
 11%|█         | 1108/10000 [1:18:57<10:28:17,  4.24s/it]
 11%|█         | 1109/10000 [1:19:01<10:12:10,  4.13s/it]
 11%|█         | 1110/10000 [1:19:06<10:25:34,  4.22s/it]
 11%|█         | 1111/10000 [1:19:10<10:33:15,  4.27s/it]
 11%|█         | 1112/10000 [1:19:14<10:31:37,  4.26s/it]
 11%|█         | 1113/10000 [1:19:18<10:24:44,  4.22s/it]
 11%|█         | 1114/10000 [1:19:23<10:37:55,  4.31s/it]
 11%|█         | 1115/10000 [1:19:27<10:29:51,  4.25s/it]
 11%|█         | 1116/10000 [1:19:31<10:25:01,  4.22s/it]
 11%|█         | 1117/10000 [1:19:36<10:33:37,  4.28s/it]
 11%|█         | 1118/10000 [1:19:40<10:26:47,  4.23s/it]
 11%|█         | 1119/10000 [1:19:44<10:20:58,  4.20s/it]
 11%|█         | 1120/10000 [1:19:48<10:18:22,  4.18s/it]
                                                         

 11%|█         | 1120/10000 [1:19:48<10:18:22,  4.18s/it]
 11%|█         | 1121/10000 [1:19:52<10:30:15,  4.26s/it]
 11%|█         | 1122/10000 [1:19:56<10:24:08,  4.22s/it]
 11%|█         | 1123/10000 [1:20:00<10:14:52,  4.16s/it]
 11%|█         | 1124/10000 [1:20:04<10:02:53,  4.08s/it]
 11%|█▏        | 1125/10000 [1:20:09<10:18:06,  4.18s/it]
 11%|█▏        | 1126/10000 [1:20:13<10:15:30,  4.16s/it]
 11%|█▏        | 1127/10000 [1:20:17<10:13:58,  4.15s/it]
 11%|█▏        | 1128/10000 [1:20:21<10:27:21,  4.24s/it]
 11%|█▏        | 1129/10000 [1:20:26<10:28:47,  4.25s/it]
 11%|█▏        | 1130/10000 [1:20:30<10:22:50,  4.21s/it]
 11%|█▏        | 1131/10000 [1:20:34<10:19:30,  4.19s/it]
 11%|█▏        | 1132/10000 [1:20:38<10:30:31,  4.27s/it]
 11%|█▏        | 1133/10000 [1:20:43<10:27:09,  4.24s/it]
 11%|█▏        | 1134/10000 [1:20:47<10:23:18,  4.22s/it]
 11%|█▏        | 1135/10000 [1:20:51<10:33:33,  4.29s/it]
 11%|█▏        | 1136/10000 [1:20:55<10:26:05,  4.24s/it]
 11%|█▏        | 1137/10000 [1:21:00<10:22:25,  4.21s/it]
 11%|█▏        | 1138/10000 [1:21:04<10:19:01,  4.19s/it]
 11%|█▏        | 1139/10000 [1:21:08<10:36:22,  4.31s/it]
 11%|█▏        | 1140/10000 [1:21:12<10:28:22,  4.26s/it]
                                                         

 11%|█▏        | 1140/10000 [1:21:12<10:28:22,  4.26s/it]
 11%|█▏        | 1141/10000 [1:21:17<10:22:25,  4.22s/it]
 11%|█▏        | 1142/10000 [1:21:21<10:32:09,  4.28s/it]
 11%|█▏        | 1143/10000 [1:21:25<10:25:20,  4.24s/it]
 11%|█▏        | 1144/10000 [1:21:29<10:24:03,  4.23s/it]
 11%|█▏        | 1145/10000 [1:21:33<10:20:02,  4.20s/it]
 11%|█▏        | 1146/10000 [1:21:38<10:32:17,  4.28s/it]
 11%|█▏        | 1147/10000 [1:21:42<10:29:48,  4.27s/it]
 11%|█▏        | 1148/10000 [1:21:46<10:22:40,  4.22s/it]
 11%|█▏        | 1149/10000 [1:21:51<10:32:08,  4.29s/it]
 12%|█▏        | 1150/10000 [1:21:55<10:25:56,  4.24s/it]
 12%|█▏        | 1151/10000 [1:21:59<10:31:21,  4.28s/it]
 12%|█▏        | 1152/10000 [1:22:03<10:25:09,  4.24s/it]
 12%|█▏        | 1153/10000 [1:22:08<10:34:11,  4.30s/it]
 12%|█▏        | 1154/10000 [1:22:12<10:29:49,  4.27s/it]
 12%|█▏        | 1155/10000 [1:22:16<10:24:20,  4.24s/it]
 12%|█▏        | 1156/10000 [1:22:21<10:37:59,  4.33s/it]
 12%|█▏        | 1157/10000 [1:22:25<10:40:36,  4.35s/it]
 12%|█▏        | 1158/10000 [1:22:29<10:31:44,  4.29s/it]
 12%|█▏        | 1159/10000 [1:22:33<10:25:15,  4.24s/it]
 12%|█▏        | 1160/10000 [1:22:38<10:23:13,  4.23s/it]
                                                         

 12%|█▏        | 1160/10000 [1:22:38<10:23:13,  4.23s/it]
 12%|█▏        | 1161/10000 [1:22:41<10:09:58,  4.14s/it]
 12%|█▏        | 1162/10000 [1:22:46<10:10:09,  4.14s/it]
 12%|█▏        | 1163/10000 [1:22:50<10:24:50,  4.24s/it]
 12%|█▏        | 1164/10000 [1:22:54<10:20:23,  4.21s/it]
 12%|█▏        | 1165/10000 [1:22:58<10:17:24,  4.19s/it]
 12%|█▏        | 1166/10000 [1:23:03<10:14:52,  4.18s/it]
 12%|█▏        | 1167/10000 [1:23:07<10:28:57,  4.27s/it]
 12%|█▏        | 1168/10000 [1:23:11<10:23:36,  4.24s/it]
 12%|█▏        | 1169/10000 [1:23:15<10:20:51,  4.22s/it]
 12%|█▏        | 1170/10000 [1:23:20<10:32:10,  4.30s/it]
 12%|█▏        | 1171/10000 [1:23:24<10:20:29,  4.22s/it]
 12%|█▏        | 1172/10000 [1:23:28<10:17:38,  4.20s/it]
 12%|█▏        | 1173/10000 [1:23:32<10:16:16,  4.19s/it]
 12%|█▏        | 1174/10000 [1:23:36<10:17:59,  4.20s/it]
 12%|█▏        | 1175/10000 [1:23:41<10:24:46,  4.25s/it]
 12%|█▏        | 1176/10000 [1:23:45<10:19:14,  4.21s/it]
 12%|█▏        | 1177/10000 [1:23:49<10:31:44,  4.30s/it]
 12%|█▏        | 1178/10000 [1:23:54<10:24:24,  4.25s/it]
 12%|█▏        | 1179/10000 [1:23:58<10:15:34,  4.19s/it]
 12%|█▏        | 1180/10000 [1:24:02<10:11:35,  4.16s/it]
                                                         

 12%|█▏        | 1180/10000 [1:24:02<10:11:35,  4.16s/it]
 12%|█▏        | 1181/10000 [1:24:06<10:22:49,  4.24s/it]
 12%|█▏        | 1182/10000 [1:24:10<10:17:07,  4.20s/it]
 12%|█▏        | 1183/10000 [1:24:14<10:13:32,  4.18s/it]
 12%|█▏        | 1184/10000 [1:24:19<10:24:01,  4.25s/it]
 12%|█▏        | 1185/10000 [1:24:23<10:20:10,  4.22s/it]
 12%|█▏        | 1186/10000 [1:24:27<10:17:56,  4.21s/it]
 12%|█▏        | 1187/10000 [1:24:31<10:15:42,  4.19s/it]
 12%|█▏        | 1188/10000 [1:24:36<10:27:16,  4.27s/it]
 12%|█▏        | 1189/10000 [1:24:40<10:22:30,  4.24s/it]
 12%|█▏        | 1190/10000 [1:24:44<10:18:37,  4.21s/it]
 12%|█▏        | 1191/10000 [1:24:48<10:25:55,  4.26s/it]
 12%|█▏        | 1192/10000 [1:24:53<10:23:19,  4.25s/it]
 12%|█▏        | 1193/10000 [1:24:57<10:08:23,  4.14s/it]
 12%|█▏        | 1194/10000 [1:25:01<10:09:14,  4.15s/it]
 12%|█▏        | 1195/10000 [1:25:05<10:23:18,  4.25s/it]
 12%|█▏        | 1196/10000 [1:25:10<10:38:21,  4.35s/it]
 12%|█▏        | 1197/10000 [1:25:14<10:38:35,  4.35s/it]
 12%|█▏        | 1198/10000 [1:25:18<10:33:14,  4.32s/it]
 12%|█▏        | 1199/10000 [1:25:22<10:26:23,  4.27s/it]
 12%|█▏        | 1200/10000 [1:25:27<10:21:05,  4.23s/it]
                                                         

 12%|█▏        | 1200/10000 [1:25:27<10:21:05,  4.23s/it]
 12%|█▏        | 1201/10000 [1:25:31<10:17:45,  4.21s/it]
 12%|█▏        | 1202/10000 [1:25:35<10:18:19,  4.22s/it]
 12%|█▏        | 1203/10000 [1:25:39<10:22:43,  4.25s/it]
 12%|█▏        | 1204/10000 [1:25:44<10:20:33,  4.23s/it]
 12%|█▏        | 1205/10000 [1:25:48<10:30:19,  4.30s/it]
 12%|█▏        | 1206/10000 [1:25:52<10:24:01,  4.26s/it]
 12%|█▏        | 1207/10000 [1:25:56<10:20:14,  4.23s/it]
 12%|█▏        | 1208/10000 [1:26:01<10:18:19,  4.22s/it]
 12%|█▏        | 1209/10000 [1:26:05<10:29:04,  4.29s/it]
 12%|█▏        | 1210/10000 [1:26:09<10:23:25,  4.26s/it]
 12%|█▏        | 1211/10000 [1:26:13<10:20:22,  4.24s/it]
 12%|█▏        | 1212/10000 [1:26:18<10:29:11,  4.30s/it]
 12%|█▏        | 1213/10000 [1:26:22<10:22:47,  4.25s/it]
 12%|█▏        | 1214/10000 [1:26:26<10:26:49,  4.28s/it]
 12%|█▏        | 1215/10000 [1:26:30<10:20:01,  4.23s/it]
 12%|█▏        | 1216/10000 [1:26:35<10:44:39,  4.40s/it]
 12%|█▏        | 1217/10000 [1:26:39<10:34:03,  4.33s/it]
 12%|█▏        | 1218/10000 [1:26:44<10:25:57,  4.28s/it]
 12%|█▏        | 1219/10000 [1:26:48<10:35:27,  4.34s/it]
 12%|█▏        | 1220/10000 [1:26:52<10:26:51,  4.28s/it]
                                                         

 12%|█▏        | 1220/10000 [1:26:52<10:26:51,  4.28s/it]
 12%|█▏        | 1221/10000 [1:26:56<10:21:10,  4.25s/it]
 12%|█▏        | 1222/10000 [1:27:01<10:26:11,  4.28s/it]
 12%|█▏        | 1223/10000 [1:27:05<10:32:47,  4.33s/it]
 12%|█▏        | 1224/10000 [1:27:09<10:16:53,  4.22s/it]
 12%|█▏        | 1225/10000 [1:27:13<10:21:43,  4.25s/it]
 12%|█▏        | 1226/10000 [1:27:18<10:48:00,  4.43s/it]
 12%|█▏        | 1227/10000 [1:27:22<10:35:39,  4.35s/it]
 12%|█▏        | 1228/10000 [1:27:26<10:17:10,  4.22s/it]
 12%|█▏        | 1229/10000 [1:27:30<10:13:16,  4.20s/it]
 12%|█▏        | 1230/10000 [1:27:35<10:24:48,  4.27s/it]
 12%|█▏        | 1231/10000 [1:27:39<10:19:46,  4.24s/it]
 12%|█▏        | 1232/10000 [1:27:43<10:09:46,  4.17s/it]
 12%|█▏        | 1233/10000 [1:27:47<10:15:27,  4.21s/it]
 12%|█▏        | 1234/10000 [1:27:51<10:07:10,  4.16s/it]
 12%|█▏        | 1235/10000 [1:27:55<10:01:13,  4.12s/it]
 12%|█▏        | 1236/10000 [1:27:59<9:57:58,  4.09s/it] 
 12%|█▏        | 1237/10000 [1:28:04<10:07:14,  4.16s/it]
 12%|█▏        | 1238/10000 [1:28:08<10:03:05,  4.13s/it]
 12%|█▏        | 1239/10000 [1:28:12<10:00:39,  4.11s/it]
 12%|█▏        | 1240/10000 [1:28:16<10:12:42,  4.20s/it]
                                                         

 12%|█▏        | 1240/10000 [1:28:16<10:12:42,  4.20s/it]
 12%|█▏        | 1241/10000 [1:28:20<10:07:44,  4.16s/it]
 12%|█▏        | 1242/10000 [1:28:24<10:02:45,  4.13s/it]
 12%|█▏        | 1243/10000 [1:28:28<9:57:55,  4.10s/it] 
 12%|█▏        | 1244/10000 [1:28:33<10:07:03,  4.16s/it]
 12%|█▏        | 1245/10000 [1:28:37<10:00:25,  4.11s/it]
 12%|█▏        | 1246/10000 [1:28:41<9:56:33,  4.09s/it] 
 12%|█▏        | 1247/10000 [1:28:45<10:05:22,  4.15s/it]
 12%|█▏        | 1248/10000 [1:28:49<10:00:59,  4.12s/it]
 12%|█▏        | 1249/10000 [1:28:53<9:57:30,  4.10s/it] 
 12%|█▎        | 1250/10000 [1:28:57<9:53:55,  4.07s/it]
 13%|█▎        | 1251/10000 [1:29:02<10:04:44,  4.15s/it]
 13%|█▎        | 1252/10000 [1:29:06<9:59:17,  4.11s/it] 
 13%|█▎        | 1253/10000 [1:29:10<9:56:01,  4.09s/it]
 13%|█▎        | 1254/10000 [1:29:14<9:53:09,  4.07s/it]
 13%|█▎        | 1255/10000 [1:29:18<9:54:09,  4.08s/it]
 13%|█▎        | 1256/10000 [1:29:22<9:57:06,  4.10s/it]
 13%|█▎        | 1257/10000 [1:29:26<9:53:53,  4.08s/it]
 13%|█▎        | 1258/10000 [1:29:30<10:05:32,  4.16s/it]
 13%|█▎        | 1259/10000 [1:29:34<10:05:34,  4.16s/it]
 13%|█▎        | 1260/10000 [1:29:39<10:13:39,  4.21s/it]
                                                         

 13%|█▎        | 1260/10000 [1:29:39<10:13:39,  4.21s/it]
 13%|█▎        | 1261/10000 [1:29:43<10:19:48,  4.26s/it]
 13%|█▎        | 1262/10000 [1:29:48<10:34:37,  4.36s/it]
 13%|█▎        | 1263/10000 [1:29:52<10:26:31,  4.30s/it]
 13%|█▎        | 1264/10000 [1:29:56<10:35:10,  4.36s/it]
 13%|█▎        | 1265/10000 [1:30:01<10:38:03,  4.38s/it]
 13%|█▎        | 1266/10000 [1:30:05<10:30:10,  4.33s/it]
 13%|█▎        | 1267/10000 [1:30:09<10:30:27,  4.33s/it]
 13%|█▎        | 1268/10000 [1:30:14<10:23:48,  4.29s/it]
 13%|█▎        | 1269/10000 [1:30:18<10:32:27,  4.35s/it]
 13%|█▎        | 1270/10000 [1:30:22<10:24:21,  4.29s/it]
 13%|█▎        | 1271/10000 [1:30:27<10:27:11,  4.31s/it]
 13%|█▎        | 1272/10000 [1:30:31<10:30:10,  4.33s/it]
 13%|█▎        | 1273/10000 [1:30:35<10:22:54,  4.28s/it]
 13%|█▎        | 1274/10000 [1:30:39<10:23:04,  4.28s/it]
 13%|█▎        | 1275/10000 [1:30:44<10:21:45,  4.28s/it]
 13%|█▎        | 1276/10000 [1:30:48<10:29:13,  4.33s/it]
 13%|█▎        | 1277/10000 [1:30:52<10:22:03,  4.28s/it]
 13%|█▎        | 1278/10000 [1:30:57<10:22:06,  4.28s/it]
 13%|█▎        | 1279/10000 [1:31:01<10:26:15,  4.31s/it]
 13%|█▎        | 1280/10000 [1:31:05<10:15:56,  4.24s/it]
                                                         
{'loss': '0.06393', 'grad_norm': '0.0002631', 'learning_rate': '4.45e-06', 'num_tokens': '3.274e+05', 'completions/mean_length': '4.219', 'completions/min_length': '3.65', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.219', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.287', 'rewards/reward_semantic/std': '0.03753', 'rewards/reward_length/mean': '-0.03342', 'rewards/reward_length/std': '0.0102', 'reward': '1.084', 'reward_std': '0.01162', 'frac_reward_zero_std': '0.6', 'kl': '1.532', 'entropy': '0.1959', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.215', 'epoch': '0.1804'}
{'loss': '0.07068', 'grad_norm': '0.04787', 'learning_rate': '4.441e-06', 'num_tokens': '3.33e+05', 'completions/mean_length': '4.456', 'completions/min_length': '3.95', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.456', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.164', 'rewards/reward_semantic/std': '0.02239', 'rewards/reward_length/mean': '-0.03203', 'rewards/reward_length/std': '0.007366', 'reward': '1.048', 'reward_std': '0.006921', 'frac_reward_zero_std': '0.6', 'kl': '1.84', 'entropy': '0.321', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.204', 'epoch': '0.1837'}
{'loss': '0.1098', 'grad_norm': '0.0007943', 'learning_rate': '4.431e-06', 'num_tokens': '3.389e+05', 'completions/mean_length': '4.631', 'completions/min_length': '4.5', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.631', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2', 'rewards/reward_semantic/std': '0', 'rewards/reward_length/mean': '-0.021', 'rewards/reward_length/std': '0.00172', 'reward': '1.059', 'reward_std': '8.6e-05', 'frac_reward_zero_std': '0.85', 'kl': '2.608', 'entropy': '0.09031', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.205', 'epoch': '0.187'}
{'loss': '0.1312', 'grad_norm': '0.3228', 'learning_rate': '4.421e-06', 'num_tokens': '3.446e+05', 'completions/mean_length': '4.525', 'completions/min_length': '4.2', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.525', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2562', 'rewards/reward_semantic/std': '0.06123', 'rewards/reward_length/mean': '-0.02017', 'rewards/reward_length/std': '0.002156', 'reward': '1.076', 'reward_std': '0.01836', 'frac_reward_zero_std': '0.65', 'kl': '2.79', 'entropy': '0.1005', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.239', 'epoch': '0.1903'}
{'loss': '0.1155', 'grad_norm': '0.01764', 'learning_rate': '4.411e-06', 'num_tokens': '3.504e+05', 'completions/mean_length': '4.419', 'completions/min_length': '4.05', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.419', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1812', 'rewards/reward_semantic/std': '0.06826', 'rewards/reward_length/mean': '-0.02422', 'rewards/reward_length/std': '0.005715', 'reward': '1.053', 'reward_std': '0.02054', 'frac_reward_zero_std': '0.65', 'kl': '2.199', 'entropy': '0.105', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.186', 'epoch': '0.1936'}
{'loss': '0.09674', 'grad_norm': '0.00645', 'learning_rate': '4.401e-06', 'num_tokens': '3.565e+05', 'completions/mean_length': '4.9', 'completions/min_length': '4.8', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.9', 'completions/min_terminated_length': '4.8', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3333', 'rewards/reward_semantic/std': '0.01782', 'rewards/reward_length/mean': '-0.02295', 'rewards/reward_length/std': '0.001423', 'reward': '1.099', 'reward_std': '0.005416', 'frac_reward_zero_std': '0.85', 'kl': '2.595', 'entropy': '0.06225', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.228', 'epoch': '0.1969'}
{'loss': '0.1487', 'grad_norm': '0.008373', 'learning_rate': '4.391e-06', 'num_tokens': '3.626e+05', 'completions/mean_length': '4.85', 'completions/min_length': '4.35', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.85', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.09792', 'rewards/reward_semantic/std': '0.05829', 'rewards/reward_length/mean': '-0.025', 'rewards/reward_length/std': '0.007985', 'reward': '1.028', 'reward_std': '0.0175', 'frac_reward_zero_std': '0.55', 'kl': '3.114', 'entropy': '0.1317', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.256', 'epoch': '0.2001'}
{'loss': '0.07973', 'grad_norm': '11.05', 'learning_rate': '4.381e-06', 'num_tokens': '3.684e+05', 'completions/mean_length': '4.656', 'completions/min_length': '4.35', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.656', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2234', 'rewards/reward_semantic/std': '0.07564', 'rewards/reward_length/mean': '-0.02112', 'rewards/reward_length/std': '0.007867', 'reward': '1.066', 'reward_std': '0.02252', 'frac_reward_zero_std': '0.65', 'kl': '3.016', 'entropy': '0.1108', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.19', 'epoch': '0.2034'}
{'loss': '0.1165', 'grad_norm': '12.04', 'learning_rate': '4.37e-06', 'num_tokens': '3.745e+05', 'completions/mean_length': '4.787', 'completions/min_length': '4.6', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.787', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1562', 'rewards/reward_semantic/std': '0.05303', 'rewards/reward_length/mean': '-0.02009', 'rewards/reward_length/std': '0.004627', 'reward': '1.046', 'reward_std': '0.0158', 'frac_reward_zero_std': '0.75', 'kl': '2.853', 'entropy': '0.05656', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.102', 'epoch': '0.2067'}

 13%|█▎        | 1280/10000 [1:31:05<10:15:56,  4.24s/it]
 13%|█▎        | 1281/10000 [1:31:09<10:10:38,  4.20s/it]
 13%|█▎        | 1282/10000 [1:31:14<10:21:31,  4.28s/it]
 13%|█▎        | 1283/10000 [1:31:18<10:11:31,  4.21s/it]
 13%|█▎        | 1284/10000 [1:31:22<10:04:34,  4.16s/it]
 13%|█▎        | 1285/10000 [1:31:26<9:59:00,  4.12s/it] 
 13%|█▎        | 1286/10000 [1:31:30<10:07:32,  4.18s/it]
 13%|█▎        | 1287/10000 [1:31:34<10:01:18,  4.14s/it]
 13%|█▎        | 1288/10000 [1:31:38<9:56:52,  4.11s/it] 
 13%|█▎        | 1289/10000 [1:31:42<9:56:01,  4.11s/it]
 13%|█▎        | 1290/10000 [1:31:46<9:48:54,  4.06s/it]
 13%|█▎        | 1291/10000 [1:31:50<9:59:16,  4.13s/it]
 13%|█▎        | 1292/10000 [1:31:55<10:01:19,  4.14s/it]
 13%|█▎        | 1293/10000 [1:32:00<10:43:14,  4.43s/it]
 13%|█▎        | 1294/10000 [1:32:04<10:23:26,  4.30s/it]
 13%|█▎        | 1295/10000 [1:32:07<10:02:28,  4.15s/it]
 13%|█▎        | 1296/10000 [1:32:12<10:10:40,  4.21s/it]
 13%|█▎        | 1297/10000 [1:32:16<10:03:46,  4.16s/it]
 13%|█▎        | 1298/10000 [1:32:20<9:52:48,  4.09s/it] 
 13%|█▎        | 1299/10000 [1:32:24<9:41:27,  4.01s/it]
 13%|█▎        | 1300/10000 [1:32:28<10:01:31,  4.15s/it]
                                                         

 13%|█▎        | 1300/10000 [1:32:28<10:01:31,  4.15s/it]
 13%|█▎        | 1301/10000 [1:32:32<9:52:14,  4.08s/it] 
 13%|█▎        | 1302/10000 [1:32:36<9:50:24,  4.07s/it]
 13%|█▎        | 1303/10000 [1:32:41<10:13:57,  4.24s/it]
 13%|█▎        | 1304/10000 [1:32:45<10:09:08,  4.20s/it]
 13%|█▎        | 1305/10000 [1:32:49<10:01:57,  4.15s/it]
 13%|█▎        | 1306/10000 [1:32:53<9:56:00,  4.11s/it] 
 13%|█▎        | 1307/10000 [1:32:57<10:02:50,  4.16s/it]
 13%|█▎        | 1308/10000 [1:33:01<10:01:54,  4.15s/it]
 13%|█▎        | 1309/10000 [1:33:05<10:01:06,  4.15s/it]
 13%|█▎        | 1310/10000 [1:33:10<10:27:33,  4.33s/it]
 13%|█▎        | 1311/10000 [1:33:14<10:18:50,  4.27s/it]
 13%|█▎        | 1312/10000 [1:33:18<10:12:10,  4.23s/it]
 13%|█▎        | 1313/10000 [1:33:23<10:08:39,  4.20s/it]
 13%|█▎        | 1314/10000 [1:33:27<10:18:54,  4.28s/it]
 13%|█▎        | 1315/10000 [1:33:31<10:23:10,  4.31s/it]
 13%|█▎        | 1316/10000 [1:33:36<10:21:14,  4.29s/it]
 13%|█▎        | 1317/10000 [1:33:40<10:17:15,  4.27s/it]
 13%|█▎        | 1318/10000 [1:33:44<10:01:54,  4.16s/it]
 13%|█▎        | 1319/10000 [1:33:48<10:08:19,  4.20s/it]
 13%|█▎        | 1320/10000 [1:33:52<10:09:34,  4.21s/it]
                                                         

 13%|█▎        | 1320/10000 [1:33:52<10:09:34,  4.21s/it]
 13%|█▎        | 1321/10000 [1:33:56<10:05:51,  4.19s/it]
 13%|█▎        | 1322/10000 [1:34:01<10:03:56,  4.18s/it]
 13%|█▎        | 1323/10000 [1:34:05<10:02:30,  4.17s/it]
 13%|█▎        | 1324/10000 [1:34:09<10:14:06,  4.25s/it]
 13%|█▎        | 1325/10000 [1:34:13<10:00:15,  4.15s/it]
 13%|█▎        | 1326/10000 [1:34:17<9:59:07,  4.14s/it] 
 13%|█▎        | 1327/10000 [1:34:22<10:09:37,  4.22s/it]
 13%|█▎        | 1328/10000 [1:34:26<10:03:53,  4.18s/it]
 13%|█▎        | 1329/10000 [1:34:30<9:59:09,  4.15s/it] 
 13%|█▎        | 1330/10000 [1:34:34<10:13:48,  4.25s/it]
 13%|█▎        | 1331/10000 [1:34:38<10:09:48,  4.22s/it]
 13%|█▎        | 1332/10000 [1:34:43<10:06:23,  4.20s/it]
 13%|█▎        | 1333/10000 [1:34:47<10:09:22,  4.22s/it]
 13%|█▎        | 1334/10000 [1:34:51<10:18:17,  4.28s/it]
 13%|█▎        | 1335/10000 [1:34:55<10:09:25,  4.22s/it]
 13%|█▎        | 1336/10000 [1:34:59<9:57:46,  4.14s/it] 
 13%|█▎        | 1337/10000 [1:35:04<10:30:33,  4.37s/it]
 13%|█▎        | 1338/10000 [1:35:08<10:22:32,  4.31s/it]
 13%|█▎        | 1339/10000 [1:35:12<10:09:30,  4.22s/it]
 13%|█▎        | 1340/10000 [1:35:17<10:20:34,  4.30s/it]
                                                         

 13%|█▎        | 1340/10000 [1:35:17<10:20:34,  4.30s/it]
 13%|█▎        | 1341/10000 [1:35:21<10:17:24,  4.28s/it]
 13%|█▎        | 1342/10000 [1:35:25<10:13:28,  4.25s/it]
 13%|█▎        | 1343/10000 [1:35:30<10:16:13,  4.27s/it]
 13%|█▎        | 1344/10000 [1:35:34<10:10:23,  4.23s/it]
 13%|█▎        | 1345/10000 [1:35:38<10:22:39,  4.32s/it]
 13%|█▎        | 1346/10000 [1:35:43<10:24:20,  4.33s/it]
 13%|█▎        | 1347/10000 [1:35:47<10:30:42,  4.37s/it]
 13%|█▎        | 1348/10000 [1:35:51<10:16:48,  4.28s/it]
 13%|█▎        | 1349/10000 [1:35:55<10:07:44,  4.22s/it]
 14%|█▎        | 1350/10000 [1:36:00<10:15:11,  4.27s/it]
 14%|█▎        | 1351/10000 [1:36:04<10:20:30,  4.30s/it]
 14%|█▎        | 1352/10000 [1:36:08<10:13:54,  4.26s/it]
 14%|█▎        | 1353/10000 [1:36:13<10:21:54,  4.32s/it]
 14%|█▎        | 1354/10000 [1:36:17<10:15:04,  4.27s/it]
 14%|█▎        | 1355/10000 [1:36:21<9:59:27,  4.16s/it] 
 14%|█▎        | 1356/10000 [1:36:25<9:58:38,  4.16s/it]
 14%|█▎        | 1357/10000 [1:36:29<10:13:24,  4.26s/it]
 14%|█▎        | 1358/10000 [1:36:33<9:59:12,  4.16s/it] 
 14%|█▎        | 1359/10000 [1:36:37<9:58:32,  4.16s/it]
 14%|█▎        | 1360/10000 [1:36:42<10:21:43,  4.32s/it]
                                                         

 14%|█▎        | 1360/10000 [1:36:42<10:21:43,  4.32s/it]
 14%|█▎        | 1361/10000 [1:36:46<10:13:38,  4.26s/it]
 14%|█▎        | 1362/10000 [1:36:51<10:21:50,  4.32s/it]
 14%|█▎        | 1363/10000 [1:36:56<10:45:21,  4.48s/it]
 14%|█▎        | 1364/10000 [1:37:00<10:30:40,  4.38s/it]
 14%|█▎        | 1365/10000 [1:37:04<10:14:29,  4.27s/it]
 14%|█▎        | 1366/10000 [1:37:08<10:30:47,  4.38s/it]
 14%|█▎        | 1367/10000 [1:37:12<10:12:47,  4.26s/it]
 14%|█▎        | 1368/10000 [1:37:16<10:08:33,  4.23s/it]
 14%|█▎        | 1369/10000 [1:37:21<10:06:12,  4.21s/it]
 14%|█▎        | 1370/10000 [1:37:25<10:20:38,  4.31s/it]
 14%|█▎        | 1371/10000 [1:37:29<10:14:43,  4.27s/it]
 14%|█▎        | 1372/10000 [1:37:34<10:22:41,  4.33s/it]
 14%|█▎        | 1373/10000 [1:37:38<10:32:44,  4.40s/it]
 14%|█▎        | 1374/10000 [1:37:43<10:55:03,  4.56s/it]
 14%|█▍        | 1375/10000 [1:37:47<10:38:05,  4.44s/it]
 14%|█▍        | 1376/10000 [1:37:52<10:56:43,  4.57s/it]
 14%|█▍        | 1377/10000 [1:37:57<10:44:05,  4.48s/it]
 14%|█▍        | 1378/10000 [1:38:01<10:42:25,  4.47s/it]
 14%|█▍        | 1379/10000 [1:38:05<10:28:47,  4.38s/it]
 14%|█▍        | 1380/10000 [1:38:10<10:32:17,  4.40s/it]
                                                         

 14%|█▍        | 1380/10000 [1:38:10<10:32:17,  4.40s/it]
 14%|█▍        | 1381/10000 [1:38:14<10:21:38,  4.33s/it]
 14%|█▍        | 1382/10000 [1:38:18<10:20:13,  4.32s/it]
 14%|█▍        | 1383/10000 [1:38:23<10:26:20,  4.36s/it]
 14%|█▍        | 1384/10000 [1:38:27<10:20:19,  4.32s/it]
 14%|█▍        | 1385/10000 [1:38:31<10:14:14,  4.28s/it]
 14%|█▍        | 1386/10000 [1:38:36<10:31:01,  4.40s/it]
 14%|█▍        | 1387/10000 [1:38:40<10:21:47,  4.33s/it]
 14%|█▍        | 1388/10000 [1:38:44<10:20:05,  4.32s/it]
 14%|█▍        | 1389/10000 [1:38:48<10:15:53,  4.29s/it]
 14%|█▍        | 1390/10000 [1:38:53<10:10:18,  4.25s/it]
 14%|█▍        | 1391/10000 [1:38:57<10:06:36,  4.23s/it]
 14%|█▍        | 1392/10000 [1:39:01<10:11:56,  4.27s/it]
 14%|█▍        | 1393/10000 [1:39:06<10:18:52,  4.31s/it]
 14%|█▍        | 1394/10000 [1:39:10<10:09:43,  4.25s/it]
 14%|█▍        | 1395/10000 [1:39:14<10:00:43,  4.19s/it]
 14%|█▍        | 1396/10000 [1:39:18<10:17:54,  4.31s/it]
 14%|█▍        | 1397/10000 [1:39:22<10:05:23,  4.22s/it]
 14%|█▍        | 1398/10000 [1:39:26<9:57:28,  4.17s/it] 
 14%|█▍        | 1399/10000 [1:39:31<10:11:41,  4.27s/it]
 14%|█▍        | 1400/10000 [1:39:35<9:53:05,  4.14s/it] 
                                                        

 14%|█▍        | 1400/10000 [1:39:35<9:53:05,  4.14s/it]
 14%|█▍        | 1401/10000 [1:39:39<10:01:06,  4.19s/it]
 14%|█▍        | 1402/10000 [1:39:43<10:03:37,  4.21s/it]
 14%|█▍        | 1403/10000 [1:39:47<9:57:18,  4.17s/it] 
 14%|█▍        | 1404/10000 [1:39:51<9:41:21,  4.06s/it]
 14%|█▍        | 1405/10000 [1:39:55<9:31:34,  3.99s/it]
 14%|█▍        | 1406/10000 [1:39:59<9:52:13,  4.13s/it]
 14%|█▍        | 1407/10000 [1:40:03<9:49:28,  4.12s/it]
 14%|█▍        | 1408/10000 [1:40:07<9:37:41,  4.03s/it]
 14%|█▍        | 1409/10000 [1:40:12<9:50:30,  4.12s/it]
 14%|█▍        | 1410/10000 [1:40:15<9:38:28,  4.04s/it]
 14%|█▍        | 1411/10000 [1:40:20<10:02:31,  4.21s/it]
 14%|█▍        | 1412/10000 [1:40:25<10:12:28,  4.28s/it]
 14%|█▍        | 1413/10000 [1:40:29<10:17:06,  4.31s/it]
 14%|█▍        | 1414/10000 [1:40:33<10:03:07,  4.21s/it]
 14%|█▍        | 1415/10000 [1:40:37<9:46:32,  4.10s/it] 
 14%|█▍        | 1416/10000 [1:40:41<9:58:16,  4.18s/it]
 14%|█▍        | 1417/10000 [1:40:45<9:54:33,  4.16s/it]
 14%|█▍        | 1418/10000 [1:40:49<9:49:41,  4.12s/it]
 14%|█▍        | 1419/10000 [1:40:54<10:09:28,  4.26s/it]
 14%|█▍        | 1420/10000 [1:40:58<10:05:42,  4.24s/it]
                                                         

 14%|█▍        | 1420/10000 [1:40:58<10:05:42,  4.24s/it]
 14%|█▍        | 1421/10000 [1:41:02<9:48:35,  4.12s/it] 
 14%|█▍        | 1422/10000 [1:41:06<9:57:27,  4.18s/it]
 14%|█▍        | 1423/10000 [1:41:11<10:13:20,  4.29s/it]
 14%|█▍        | 1424/10000 [1:41:15<10:06:41,  4.24s/it]
 14%|█▍        | 1425/10000 [1:41:19<10:19:54,  4.34s/it]
 14%|█▍        | 1426/10000 [1:41:23<10:08:13,  4.26s/it]
 14%|█▍        | 1427/10000 [1:41:28<9:58:50,  4.19s/it] 
 14%|█▍        | 1428/10000 [1:41:31<9:46:27,  4.10s/it]
 14%|█▍        | 1429/10000 [1:41:36<9:46:58,  4.11s/it]
 14%|█▍        | 1430/10000 [1:41:40<9:43:35,  4.09s/it]
 14%|█▍        | 1431/10000 [1:41:43<9:31:42,  4.00s/it]
 14%|█▍        | 1432/10000 [1:41:48<9:45:50,  4.10s/it]
 14%|█▍        | 1433/10000 [1:41:52<9:43:08,  4.08s/it]
 14%|█▍        | 1434/10000 [1:41:56<9:38:17,  4.05s/it]
 14%|█▍        | 1435/10000 [1:42:00<9:48:52,  4.13s/it]
 14%|█▍        | 1436/10000 [1:42:04<9:45:24,  4.10s/it]
 14%|█▍        | 1437/10000 [1:42:08<9:43:48,  4.09s/it]
 14%|█▍        | 1438/10000 [1:42:12<9:54:29,  4.17s/it]
 14%|█▍        | 1439/10000 [1:42:17<9:50:06,  4.14s/it]
 14%|█▍        | 1440/10000 [1:42:21<9:46:37,  4.11s/it]
                                                        

 14%|█▍        | 1440/10000 [1:42:21<9:46:37,  4.11s/it]
 14%|█▍        | 1441/10000 [1:42:25<9:43:51,  4.09s/it]
 14%|█▍        | 1442/10000 [1:42:29<9:55:23,  4.17s/it]
 14%|█▍        | 1443/10000 [1:42:33<9:54:05,  4.17s/it]
 14%|█▍        | 1444/10000 [1:42:37<9:37:44,  4.05s/it]
 14%|█▍        | 1445/10000 [1:42:41<9:48:34,  4.13s/it]
 14%|█▍        | 1446/10000 [1:42:45<9:42:32,  4.09s/it]
 14%|█▍        | 1447/10000 [1:42:49<9:40:11,  4.07s/it]
 14%|█▍        | 1448/10000 [1:42:54<9:50:37,  4.14s/it]
 14%|█▍        | 1449/10000 [1:42:58<9:42:36,  4.09s/it]
 14%|█▍        | 1450/10000 [1:43:02<9:40:03,  4.07s/it]
 15%|█▍        | 1451/10000 [1:43:06<9:45:43,  4.11s/it]
 15%|█▍        | 1452/10000 [1:43:10<10:08:47,  4.27s/it]
 15%|█▍        | 1453/10000 [1:43:14<9:58:39,  4.20s/it] 
 15%|█▍        | 1454/10000 [1:43:19<9:58:57,  4.21s/it]
 15%|█▍        | 1455/10000 [1:43:23<9:54:03,  4.17s/it]
 15%|█▍        | 1456/10000 [1:43:27<9:51:44,  4.16s/it]
 15%|█▍        | 1457/10000 [1:43:31<9:46:51,  4.12s/it]
 15%|█▍        | 1458/10000 [1:43:35<9:57:01,  4.19s/it]
 15%|█▍        | 1459/10000 [1:43:39<9:51:24,  4.15s/it]
 15%|█▍        | 1460/10000 [1:43:43<9:46:52,  4.12s/it]
                                                        
{'loss': '0.1223', 'grad_norm': '0.001635', 'learning_rate': '4.361e-06', 'num_tokens': '3.808e+05', 'completions/mean_length': '4.7', 'completions/min_length': '4.15', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.7', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2312', 'rewards/reward_semantic/std': '0.08172', 'rewards/reward_length/mean': '-0.03235', 'rewards/reward_length/std': '0.005023', 'reward': '1.068', 'reward_std': '0.02462', 'frac_reward_zero_std': '0.55', 'kl': '2.404', 'entropy': '0.2', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.291', 'epoch': '0.21'}
{'loss': '0.1052', 'grad_norm': '10.81', 'learning_rate': '4.351e-06', 'num_tokens': '3.867e+05', 'completions/mean_length': '4.031', 'completions/min_length': '3.65', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.031', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5873', 'rewards/reward_semantic/std': '0.08876', 'rewards/reward_length/mean': '-0.03438', 'rewards/reward_length/std': '0.007532', 'reward': '1.174', 'reward_std': '0.02665', 'frac_reward_zero_std': '0.45', 'kl': '2.393', 'entropy': '0.2161', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.138', 'epoch': '0.2133'}
{'loss': '0.06865', 'grad_norm': '0.5271', 'learning_rate': '4.341e-06', 'num_tokens': '3.927e+05', 'completions/mean_length': '4.175', 'completions/min_length': '3.75', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.175', 'completions/min_terminated_length': '3.75', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4539', 'rewards/reward_semantic/std': '0.08393', 'rewards/reward_length/mean': '-0.02717', 'rewards/reward_length/std': '0.005867', 'reward': '1.135', 'reward_std': '0.02522', 'frac_reward_zero_std': '0.6', 'kl': '2.7', 'entropy': '0.2275', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.192', 'epoch': '0.2165'}
{'loss': '0.07294', 'grad_norm': '0.005256', 'learning_rate': '4.331e-06', 'num_tokens': '3.986e+05', 'completions/mean_length': '4.381', 'completions/min_length': '4.05', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.381', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3719', 'rewards/reward_semantic/std': '0.02652', 'rewards/reward_length/mean': '-0.03398', 'rewards/reward_length/std': '0.004574', 'reward': '1.11', 'reward_std': '0.008095', 'frac_reward_zero_std': '0.7', 'kl': '2.424', 'entropy': '0.1525', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.207', 'epoch': '0.2198'}
{'loss': '0.08165', 'grad_norm': '15.94', 'learning_rate': '4.321e-06', 'num_tokens': '4.041e+05', 'completions/mean_length': '4.3', 'completions/min_length': '3.85', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.3', 'completions/min_terminated_length': '3.85', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4945', 'rewards/reward_semantic/std': '0.1223', 'rewards/reward_length/mean': '-0.03726', 'rewards/reward_length/std': '0.007194', 'reward': '1.146', 'reward_std': '0.03666', 'frac_reward_zero_std': '0.55', 'kl': '1.954', 'entropy': '0.2569', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.24', 'epoch': '0.2231'}
{'loss': '0.1118', 'grad_norm': '0.06717', 'learning_rate': '4.311e-06', 'num_tokens': '4.104e+05', 'completions/mean_length': '4.625', 'completions/min_length': '3.7', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.625', 'completions/min_terminated_length': '3.7', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2556', 'rewards/reward_semantic/std': '0.09636', 'rewards/reward_length/mean': '-0.03906', 'rewards/reward_length/std': '0.01086', 'reward': '1.075', 'reward_std': '0.0289', 'frac_reward_zero_std': '0.4', 'kl': '1.89', 'entropy': '0.4341', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.36', 'epoch': '0.2264'}
{'loss': '0.04959', 'grad_norm': '30.95', 'learning_rate': '4.301e-06', 'num_tokens': '4.161e+05', 'completions/mean_length': '4.569', 'completions/min_length': '4', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.569', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2299', 'rewards/reward_semantic/std': '0.06421', 'rewards/reward_length/mean': '-0.03472', 'rewards/reward_length/std': '0.008514', 'reward': '1.067', 'reward_std': '0.0195', 'frac_reward_zero_std': '0.45', 'kl': '1.583', 'entropy': '0.2589', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.227', 'epoch': '0.2297'}
{'loss': '0.05425', 'grad_norm': '34.17', 'learning_rate': '4.29e-06', 'num_tokens': '4.218e+05', 'completions/mean_length': '3.694', 'completions/min_length': '3', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '3.694', 'completions/min_terminated_length': '3', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3563', 'rewards/reward_semantic/std': '0.1249', 'rewards/reward_length/mean': '-0.03643', 'rewards/reward_length/std': '0.01249', 'reward': '1.105', 'reward_std': '0.03758', 'frac_reward_zero_std': '0.3', 'kl': '1.412', 'entropy': '0.4809', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.151', 'epoch': '0.2329'}
{'loss': '0.1236', 'grad_norm': '0.003751', 'learning_rate': '4.28e-06', 'num_tokens': '4.277e+05', 'completions/mean_length': '4.456', 'completions/min_length': '4.05', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.456', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4025', 'rewards/reward_semantic/std': '0.007071', 'rewards/reward_length/mean': '-0.02883', 'rewards/reward_length/std': '0.004223', 'reward': '1.119', 'reward_std': '0.002277', 'frac_reward_zero_std': '0.75', 'kl': '2.389', 'entropy': '0.1065', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.112', 'epoch': '0.2362'}

 15%|█▍        | 1460/10000 [1:43:43<9:46:52,  4.12s/it]
 15%|█▍        | 1461/10000 [1:43:48<9:47:04,  4.13s/it]
 15%|█▍        | 1462/10000 [1:43:52<9:44:26,  4.11s/it]
 15%|█▍        | 1463/10000 [1:43:56<9:59:38,  4.21s/it]
 15%|█▍        | 1464/10000 [1:44:00<9:57:33,  4.20s/it]
 15%|█▍        | 1465/10000 [1:44:05<10:03:43,  4.24s/it]
 15%|█▍        | 1466/10000 [1:44:08<9:46:53,  4.13s/it] 
 15%|█▍        | 1467/10000 [1:44:12<9:43:16,  4.10s/it]
 15%|█▍        | 1468/10000 [1:44:17<9:53:25,  4.17s/it]
 15%|█▍        | 1469/10000 [1:44:21<10:03:41,  4.25s/it]
 15%|█▍        | 1470/10000 [1:44:25<9:46:21,  4.12s/it] 
 15%|█▍        | 1471/10000 [1:44:29<9:55:01,  4.19s/it]
 15%|█▍        | 1472/10000 [1:44:33<9:40:43,  4.09s/it]
 15%|█▍        | 1473/10000 [1:44:37<9:38:57,  4.07s/it]
 15%|█▍        | 1474/10000 [1:44:42<9:50:40,  4.16s/it]
 15%|█▍        | 1475/10000 [1:44:46<9:50:45,  4.16s/it]
 15%|█▍        | 1476/10000 [1:44:50<9:51:10,  4.16s/it]
 15%|█▍        | 1477/10000 [1:44:54<9:50:53,  4.16s/it]
 15%|█▍        | 1478/10000 [1:44:59<10:01:59,  4.24s/it]
 15%|█▍        | 1479/10000 [1:45:03<9:58:35,  4.21s/it] 
 15%|█▍        | 1480/10000 [1:45:07<9:46:23,  4.13s/it]
                                                        

 15%|█▍        | 1480/10000 [1:45:07<9:46:23,  4.13s/it]
 15%|█▍        | 1481/10000 [1:45:11<10:00:05,  4.23s/it]
 15%|█▍        | 1482/10000 [1:45:15<10:01:35,  4.24s/it]
 15%|█▍        | 1483/10000 [1:45:19<9:56:52,  4.20s/it] 
 15%|█▍        | 1484/10000 [1:45:24<10:06:03,  4.27s/it]
 15%|█▍        | 1485/10000 [1:45:28<9:58:11,  4.22s/it] 
 15%|█▍        | 1486/10000 [1:45:32<9:46:11,  4.13s/it]
 15%|█▍        | 1487/10000 [1:45:36<9:46:58,  4.14s/it]
 15%|█▍        | 1488/10000 [1:45:41<10:00:15,  4.23s/it]
 15%|█▍        | 1489/10000 [1:45:45<9:57:06,  4.21s/it] 
 15%|█▍        | 1490/10000 [1:45:49<9:55:09,  4.20s/it]
 15%|█▍        | 1491/10000 [1:45:53<10:05:24,  4.27s/it]
 15%|█▍        | 1492/10000 [1:45:57<10:00:25,  4.23s/it]
 15%|█▍        | 1493/10000 [1:46:02<10:02:59,  4.25s/it]
 15%|█▍        | 1494/10000 [1:46:06<10:11:15,  4.31s/it]
 15%|█▍        | 1495/10000 [1:46:10<10:04:49,  4.27s/it]
 15%|█▍        | 1496/10000 [1:46:15<9:59:53,  4.23s/it] 
 15%|█▍        | 1497/10000 [1:46:19<10:08:45,  4.30s/it]
 15%|█▍        | 1498/10000 [1:46:23<10:14:21,  4.34s/it]
 15%|█▍        | 1499/10000 [1:46:28<10:07:17,  4.29s/it]
 15%|█▌        | 1500/10000 [1:46:32<9:57:54,  4.22s/it] 
                                                        

 15%|█▌        | 1500/10000 [1:46:32<9:57:54,  4.22s/it]
 15%|█▌        | 1501/10000 [1:46:37<10:43:37,  4.54s/it]
 15%|█▌        | 1502/10000 [1:46:41<10:20:08,  4.38s/it]
 15%|█▌        | 1503/10000 [1:46:45<10:04:51,  4.27s/it]
 15%|█▌        | 1504/10000 [1:46:49<10:07:34,  4.29s/it]
 15%|█▌        | 1505/10000 [1:46:53<9:54:16,  4.20s/it] 
 15%|█▌        | 1506/10000 [1:46:57<9:53:37,  4.19s/it]
 15%|█▌        | 1507/10000 [1:47:02<10:19:13,  4.37s/it]
 15%|█▌        | 1508/10000 [1:47:06<10:08:11,  4.30s/it]
 15%|█▌        | 1509/10000 [1:47:10<9:56:55,  4.22s/it] 
 15%|█▌        | 1510/10000 [1:47:14<9:49:10,  4.16s/it]
 15%|█▌        | 1511/10000 [1:47:19<9:56:46,  4.22s/it]
 15%|█▌        | 1512/10000 [1:47:23<9:48:59,  4.16s/it]
 15%|█▌        | 1513/10000 [1:47:27<10:01:03,  4.25s/it]
 15%|█▌        | 1514/10000 [1:47:32<10:04:13,  4.27s/it]
 15%|█▌        | 1515/10000 [1:47:36<9:53:57,  4.20s/it] 
 15%|█▌        | 1516/10000 [1:47:40<9:49:20,  4.17s/it]
 15%|█▌        | 1517/10000 [1:47:44<9:58:22,  4.23s/it]
 15%|█▌        | 1518/10000 [1:47:48<9:50:01,  4.17s/it]
 15%|█▌        | 1519/10000 [1:47:52<9:43:53,  4.13s/it]
 15%|█▌        | 1520/10000 [1:47:57<9:52:47,  4.19s/it]
                                                        

 15%|█▌        | 1520/10000 [1:47:57<9:52:47,  4.19s/it]
 15%|█▌        | 1521/10000 [1:48:01<9:45:34,  4.14s/it]
 15%|█▌        | 1522/10000 [1:48:04<9:38:02,  4.09s/it]
 15%|█▌        | 1523/10000 [1:48:09<9:36:04,  4.08s/it]
 15%|█▌        | 1524/10000 [1:48:13<9:46:32,  4.15s/it]
 15%|█▌        | 1525/10000 [1:48:17<9:46:57,  4.16s/it]
 15%|█▌        | 1526/10000 [1:48:21<9:46:06,  4.15s/it]
 15%|█▌        | 1527/10000 [1:48:26<9:53:54,  4.21s/it]
 15%|█▌        | 1528/10000 [1:48:30<9:46:49,  4.16s/it]
 15%|█▌        | 1529/10000 [1:48:34<9:44:59,  4.14s/it]
 15%|█▌        | 1530/10000 [1:48:38<9:51:46,  4.19s/it]
 15%|█▌        | 1531/10000 [1:48:42<9:46:38,  4.16s/it]
 15%|█▌        | 1532/10000 [1:48:46<9:48:38,  4.17s/it]
 15%|█▌        | 1533/10000 [1:48:50<9:45:49,  4.15s/it]
 15%|█▌        | 1534/10000 [1:48:55<9:58:26,  4.24s/it]
 15%|█▌        | 1535/10000 [1:48:59<9:55:49,  4.22s/it]
 15%|█▌        | 1536/10000 [1:49:03<9:52:48,  4.20s/it]
 15%|█▌        | 1537/10000 [1:49:07<9:52:40,  4.20s/it]
 15%|█▌        | 1538/10000 [1:49:12<10:02:05,  4.27s/it]
 15%|█▌        | 1539/10000 [1:49:16<9:57:12,  4.24s/it] 
 15%|█▌        | 1540/10000 [1:49:20<10:05:46,  4.30s/it]
                                                         

 15%|█▌        | 1540/10000 [1:49:20<10:05:46,  4.30s/it]
 15%|█▌        | 1541/10000 [1:49:24<9:50:27,  4.19s/it] 
 15%|█▌        | 1542/10000 [1:49:28<9:48:30,  4.17s/it]
 15%|█▌        | 1543/10000 [1:49:33<10:00:33,  4.26s/it]
 15%|█▌        | 1544/10000 [1:49:37<9:55:38,  4.23s/it] 
 15%|█▌        | 1545/10000 [1:49:41<9:52:27,  4.20s/it]
 15%|█▌        | 1546/10000 [1:49:45<9:50:29,  4.19s/it]
 15%|█▌        | 1547/10000 [1:49:50<10:23:15,  4.42s/it]
 15%|█▌        | 1548/10000 [1:49:54<10:09:06,  4.32s/it]
 15%|█▌        | 1549/10000 [1:49:59<10:01:04,  4.27s/it]
 16%|█▌        | 1550/10000 [1:50:03<10:12:17,  4.35s/it]
 16%|█▌        | 1551/10000 [1:50:07<10:05:32,  4.30s/it]
 16%|█▌        | 1552/10000 [1:50:11<9:56:15,  4.23s/it] 
 16%|█▌        | 1553/10000 [1:50:16<10:19:28,  4.40s/it]
 16%|█▌        | 1554/10000 [1:50:20<10:10:29,  4.34s/it]
 16%|█▌        | 1555/10000 [1:50:25<10:07:57,  4.32s/it]
 16%|█▌        | 1556/10000 [1:50:29<10:04:15,  4.29s/it]
 16%|█▌        | 1557/10000 [1:50:33<9:49:35,  4.19s/it] 
 16%|█▌        | 1558/10000 [1:50:37<9:44:46,  4.16s/it]
 16%|█▌        | 1559/10000 [1:50:41<9:45:18,  4.16s/it]
 16%|█▌        | 1560/10000 [1:50:45<9:54:47,  4.23s/it]
                                                        

 16%|█▌        | 1560/10000 [1:50:45<9:54:47,  4.23s/it]
 16%|█▌        | 1561/10000 [1:50:50<9:51:04,  4.20s/it]
 16%|█▌        | 1562/10000 [1:50:54<9:51:27,  4.21s/it]
 16%|█▌        | 1563/10000 [1:50:58<9:59:07,  4.26s/it]
 16%|█▌        | 1564/10000 [1:51:02<9:46:17,  4.17s/it]
 16%|█▌        | 1565/10000 [1:51:06<9:46:47,  4.17s/it]
 16%|█▌        | 1566/10000 [1:51:11<9:59:22,  4.26s/it]
 16%|█▌        | 1567/10000 [1:51:15<9:57:25,  4.25s/it]
 16%|█▌        | 1568/10000 [1:51:19<9:53:46,  4.23s/it]
 16%|█▌        | 1569/10000 [1:51:23<9:42:31,  4.15s/it]
 16%|█▌        | 1570/10000 [1:51:28<9:56:07,  4.24s/it]
 16%|█▌        | 1571/10000 [1:51:32<9:54:09,  4.23s/it]
 16%|█▌        | 1572/10000 [1:51:36<10:10:33,  4.35s/it]
 16%|█▌        | 1573/10000 [1:51:41<10:13:25,  4.37s/it]
 16%|█▌        | 1574/10000 [1:51:45<10:02:30,  4.29s/it]
 16%|█▌        | 1575/10000 [1:51:49<9:56:11,  4.25s/it] 
 16%|█▌        | 1576/10000 [1:51:54<10:12:40,  4.36s/it]
 16%|█▌        | 1577/10000 [1:51:58<10:03:12,  4.30s/it]
 16%|█▌        | 1578/10000 [1:52:02<9:47:44,  4.19s/it] 
 16%|█▌        | 1579/10000 [1:52:06<9:58:08,  4.26s/it]
 16%|█▌        | 1580/10000 [1:52:10<9:52:51,  4.22s/it]
                                                        

 16%|█▌        | 1580/10000 [1:52:10<9:52:51,  4.22s/it]
 16%|█▌        | 1581/10000 [1:52:14<9:40:11,  4.13s/it]
 16%|█▌        | 1582/10000 [1:52:18<9:32:03,  4.08s/it]
 16%|█▌        | 1583/10000 [1:52:23<9:46:42,  4.18s/it]
 16%|█▌        | 1584/10000 [1:52:27<9:45:58,  4.18s/it]
 16%|█▌        | 1585/10000 [1:52:31<9:43:25,  4.16s/it]
 16%|█▌        | 1586/10000 [1:52:35<9:55:10,  4.24s/it]
 16%|█▌        | 1587/10000 [1:52:40<9:50:57,  4.21s/it]
 16%|█▌        | 1588/10000 [1:52:44<9:42:24,  4.15s/it]
 16%|█▌        | 1589/10000 [1:52:47<9:33:34,  4.09s/it]
 16%|█▌        | 1590/10000 [1:52:52<9:49:49,  4.21s/it]
 16%|█▌        | 1591/10000 [1:52:56<9:48:14,  4.20s/it]
 16%|█▌        | 1592/10000 [1:53:00<9:47:20,  4.19s/it]
 16%|█▌        | 1593/10000 [1:53:05<10:02:08,  4.30s/it]
 16%|█▌        | 1594/10000 [1:53:09<9:50:08,  4.21s/it] 
 16%|█▌        | 1595/10000 [1:53:13<9:51:32,  4.22s/it]
 16%|█▌        | 1596/10000 [1:53:18<10:01:32,  4.29s/it]
 16%|█▌        | 1597/10000 [1:53:22<9:55:32,  4.25s/it] 
 16%|█▌        | 1598/10000 [1:53:26<9:51:08,  4.22s/it]
 16%|█▌        | 1599/10000 [1:53:30<9:51:25,  4.22s/it]
 16%|█▌        | 1600/10000 [1:53:34<9:57:30,  4.27s/it]
                                                        

 16%|█▌        | 1600/10000 [1:53:34<9:57:30,  4.27s/it]
 16%|█▌        | 1601/10000 [1:53:39<10:13:21,  4.38s/it]
 16%|█▌        | 1602/10000 [1:53:43<10:04:33,  4.32s/it]
 16%|█▌        | 1603/10000 [1:53:48<10:11:42,  4.37s/it]
 16%|█▌        | 1604/10000 [1:53:52<10:03:08,  4.31s/it]
 16%|█▌        | 1605/10000 [1:53:56<9:57:34,  4.27s/it] 
 16%|█▌        | 1606/10000 [1:54:01<10:04:34,  4.32s/it]
 16%|█▌        | 1607/10000 [1:54:05<10:00:21,  4.29s/it]
 16%|█▌        | 1608/10000 [1:54:09<9:56:02,  4.26s/it] 
 16%|█▌        | 1609/10000 [1:54:13<10:04:56,  4.33s/it]
 16%|█▌        | 1610/10000 [1:54:18<9:58:24,  4.28s/it] 
 16%|█▌        | 1611/10000 [1:54:22<9:44:29,  4.18s/it]
 16%|█▌        | 1612/10000 [1:54:26<9:54:20,  4.25s/it]
 16%|█▌        | 1613/10000 [1:54:30<9:50:12,  4.22s/it]
 16%|█▌        | 1614/10000 [1:54:34<9:40:00,  4.15s/it]
 16%|█▌        | 1615/10000 [1:54:38<9:39:48,  4.15s/it]
 16%|█▌        | 1616/10000 [1:54:43<9:42:27,  4.17s/it]
 16%|█▌        | 1617/10000 [1:54:47<9:38:20,  4.14s/it]
 16%|█▌        | 1618/10000 [1:54:51<9:37:25,  4.13s/it]
 16%|█▌        | 1619/10000 [1:54:55<9:49:16,  4.22s/it]
 16%|█▌        | 1620/10000 [1:54:59<9:45:52,  4.19s/it]
                                                        

 16%|█▌        | 1620/10000 [1:54:59<9:45:52,  4.19s/it]
 16%|█▌        | 1621/10000 [1:55:03<9:43:39,  4.18s/it]
 16%|█▌        | 1622/10000 [1:55:08<9:53:51,  4.25s/it]
 16%|█▌        | 1623/10000 [1:55:12<9:54:02,  4.25s/it]
 16%|█▌        | 1624/10000 [1:55:16<9:51:37,  4.24s/it]
 16%|█▋        | 1625/10000 [1:55:20<9:47:55,  4.21s/it]
 16%|█▋        | 1626/10000 [1:55:25<10:01:45,  4.31s/it]
 16%|█▋        | 1627/10000 [1:55:29<9:46:01,  4.20s/it] 
 16%|█▋        | 1628/10000 [1:55:33<9:44:39,  4.19s/it]
 16%|█▋        | 1629/10000 [1:55:38<9:56:14,  4.27s/it]
 16%|█▋        | 1630/10000 [1:55:42<9:50:41,  4.23s/it]
 16%|█▋        | 1631/10000 [1:55:46<9:37:28,  4.14s/it]
 16%|█▋        | 1632/10000 [1:55:50<9:52:06,  4.25s/it]
 16%|█▋        | 1633/10000 [1:55:54<9:56:47,  4.28s/it]
 16%|█▋        | 1634/10000 [1:55:59<9:51:23,  4.24s/it]
 16%|█▋        | 1635/10000 [1:56:03<10:07:21,  4.36s/it]
 16%|█▋        | 1636/10000 [1:56:07<9:55:25,  4.27s/it] 
 16%|█▋        | 1637/10000 [1:56:11<9:50:15,  4.23s/it]
 16%|█▋        | 1638/10000 [1:56:15<9:36:54,  4.14s/it]
 16%|█▋        | 1639/10000 [1:56:20<9:39:51,  4.16s/it]
 16%|█▋        | 1640/10000 [1:56:24<9:41:11,  4.17s/it]
                                                        
{'loss': '0.1108', 'grad_norm': '9.803e-05', 'learning_rate': '4.271e-06', 'num_tokens': '4.338e+05', 'completions/mean_length': '4.562', 'completions/min_length': '4', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.562', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4188', 'rewards/reward_semantic/std': '0.06755', 'rewards/reward_length/mean': '-0.02976', 'rewards/reward_length/std': '0.007659', 'reward': '1.124', 'reward_std': '0.02022', 'frac_reward_zero_std': '0.5', 'kl': '2.671', 'entropy': '0.1653', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.122', 'epoch': '0.2395'}
{'loss': '0.105', 'grad_norm': '0.002818', 'learning_rate': '4.261e-06', 'num_tokens': '4.397e+05', 'completions/mean_length': '4.287', 'completions/min_length': '3.85', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.287', 'completions/min_terminated_length': '3.85', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4562', 'rewards/reward_semantic/std': '0.01768', 'rewards/reward_length/mean': '-0.02866', 'rewards/reward_length/std': '0.004744', 'reward': '1.135', 'reward_std': '0.005442', 'frac_reward_zero_std': '0.75', 'kl': '2.641', 'entropy': '0.1539', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.143', 'epoch': '0.2428'}
{'loss': '0.1074', 'grad_norm': '0.007315', 'learning_rate': '4.251e-06', 'num_tokens': '4.458e+05', 'completions/mean_length': '4.669', 'completions/min_length': '4.45', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.669', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.292', 'rewards/reward_semantic/std': '0.03249', 'rewards/reward_length/mean': '-0.02603', 'rewards/reward_length/std': '0.00614', 'reward': '1.086', 'reward_std': '0.009677', 'frac_reward_zero_std': '0.8', 'kl': '2.761', 'entropy': '0.1117', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.229', 'epoch': '0.2461'}
{'loss': '0.1284', 'grad_norm': '16.47', 'learning_rate': '4.241e-06', 'num_tokens': '4.52e+05', 'completions/mean_length': '4.831', 'completions/min_length': '4.35', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.831', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1811', 'rewards/reward_semantic/std': '0.09078', 'rewards/reward_length/mean': '-0.02881', 'rewards/reward_length/std': '0.009567', 'reward': '1.053', 'reward_std': '0.02723', 'frac_reward_zero_std': '0.6', 'kl': '2.633', 'entropy': '0.1779', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.176', 'epoch': '0.2493'}
{'loss': '0.1069', 'grad_norm': '26.06', 'learning_rate': '4.231e-06', 'num_tokens': '4.579e+05', 'completions/mean_length': '4.375', 'completions/min_length': '4', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.375', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.425', 'rewards/reward_semantic/std': '0.0585', 'rewards/reward_length/mean': '-0.02781', 'rewards/reward_length/std': '0.004663', 'reward': '1.126', 'reward_std': '0.01765', 'frac_reward_zero_std': '0.6', 'kl': '2.461', 'entropy': '0.1668', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.173', 'epoch': '0.2526'}
{'loss': '0.08686', 'grad_norm': '0.02935', 'learning_rate': '4.221e-06', 'num_tokens': '4.638e+05', 'completions/mean_length': '4.394', 'completions/min_length': '4', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.394', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.397', 'rewards/reward_semantic/std': '0.007394', 'rewards/reward_length/mean': '-0.03191', 'rewards/reward_length/std': '0.005449', 'reward': '1.118', 'reward_std': '0.002374', 'frac_reward_zero_std': '0.7', 'kl': '2.769', 'entropy': '0.1916', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.234', 'epoch': '0.2559'}
{'loss': '0.1246', 'grad_norm': '0.01932', 'learning_rate': '4.21e-06', 'num_tokens': '4.698e+05', 'completions/mean_length': '4.719', 'completions/min_length': '4.6', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.719', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3219', 'rewards/reward_semantic/std': '0.02652', 'rewards/reward_length/mean': '-0.02566', 'rewards/reward_length/std': '0.003618', 'reward': '1.095', 'reward_std': '0.007987', 'frac_reward_zero_std': '0.85', 'kl': '3.2', 'entropy': '0.1003', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.227', 'epoch': '0.2592'}
{'loss': '0.1235', 'grad_norm': '18.45', 'learning_rate': '4.2e-06', 'num_tokens': '4.753e+05', 'completions/mean_length': '4.094', 'completions/min_length': '3.8', 'completions/max_length': '4.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.094', 'completions/min_terminated_length': '3.8', 'completions/max_terminated_length': '4.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4171', 'rewards/reward_semantic/std': '0.03198', 'rewards/reward_length/mean': '-0.02297', 'rewards/reward_length/std': '0.00445', 'reward': '1.124', 'reward_std': '0.009517', 'frac_reward_zero_std': '0.7', 'kl': '3.247', 'entropy': '0.08966', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.186', 'epoch': '0.2625'}
{'loss': '0.1163', 'grad_norm': '0.001785', 'learning_rate': '4.19e-06', 'num_tokens': '4.813e+05', 'completions/mean_length': '4.713', 'completions/min_length': '4.35', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.713', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3525', 'rewards/reward_semantic/std': '0.007071', 'rewards/reward_length/mean': '-0.027', 'rewards/reward_length/std': '0.0007529', 'reward': '1.104', 'reward_std': '0.002152', 'frac_reward_zero_std': '0.8', 'kl': '2.697', 'entropy': '0.07899', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.219', 'epoch': '0.2657'}

 16%|█▋        | 1640/10000 [1:56:24<9:41:11,  4.17s/it]
 16%|█▋        | 1641/10000 [1:56:29<10:06:17,  4.35s/it]
 16%|█▋        | 1642/10000 [1:56:33<10:17:55,  4.44s/it]
 16%|█▋        | 1643/10000 [1:56:37<10:06:00,  4.35s/it]
 16%|█▋        | 1644/10000 [1:56:42<10:01:31,  4.32s/it]
 16%|█▋        | 1645/10000 [1:56:46<10:06:46,  4.36s/it]
 16%|█▋        | 1646/10000 [1:56:50<9:58:04,  4.30s/it] 
 16%|█▋        | 1647/10000 [1:56:54<9:51:51,  4.25s/it]
 16%|█▋        | 1648/10000 [1:56:59<9:59:41,  4.31s/it]
 16%|█▋        | 1649/10000 [1:57:03<9:53:33,  4.26s/it]
 16%|█▋        | 1650/10000 [1:57:07<9:48:34,  4.23s/it]
 17%|█▋        | 1651/10000 [1:57:12<10:06:58,  4.36s/it]
 17%|█▋        | 1652/10000 [1:57:16<10:10:15,  4.39s/it]
 17%|█▋        | 1653/10000 [1:57:20<10:02:40,  4.33s/it]
 17%|█▋        | 1654/10000 [1:57:25<9:59:11,  4.31s/it] 
 17%|█▋        | 1655/10000 [1:57:29<10:04:39,  4.35s/it]
 17%|█▋        | 1656/10000 [1:57:33<9:56:15,  4.29s/it] 
 17%|█▋        | 1657/10000 [1:57:37<9:50:01,  4.24s/it]
 17%|█▋        | 1658/10000 [1:57:42<9:56:30,  4.29s/it]
 17%|█▋        | 1659/10000 [1:57:46<9:50:21,  4.25s/it]
 17%|█▋        | 1660/10000 [1:57:50<9:48:47,  4.24s/it]
                                                        

 17%|█▋        | 1660/10000 [1:57:50<9:48:47,  4.24s/it]
 17%|█▋        | 1661/10000 [1:57:55<9:58:34,  4.31s/it]
 17%|█▋        | 1662/10000 [1:57:59<9:55:44,  4.29s/it]
 17%|█▋        | 1663/10000 [1:58:03<9:50:38,  4.25s/it]
 17%|█▋        | 1664/10000 [1:58:08<9:59:43,  4.32s/it]
 17%|█▋        | 1665/10000 [1:58:12<9:51:50,  4.26s/it]
 17%|█▋        | 1666/10000 [1:58:16<10:02:10,  4.34s/it]
 17%|█▋        | 1667/10000 [1:58:20<9:44:45,  4.21s/it] 
 17%|█▋        | 1668/10000 [1:58:24<9:53:32,  4.27s/it]
 17%|█▋        | 1669/10000 [1:58:29<9:47:43,  4.23s/it]
 17%|█▋        | 1670/10000 [1:58:33<9:43:04,  4.20s/it]
 17%|█▋        | 1671/10000 [1:58:37<9:52:16,  4.27s/it]
 17%|█▋        | 1672/10000 [1:58:41<9:46:37,  4.23s/it]
 17%|█▋        | 1673/10000 [1:58:45<9:44:33,  4.21s/it]
 17%|█▋        | 1674/10000 [1:58:50<9:55:13,  4.29s/it]
 17%|█▋        | 1675/10000 [1:58:54<9:50:04,  4.25s/it]
 17%|█▋        | 1676/10000 [1:58:58<9:47:09,  4.23s/it]
 17%|█▋        | 1677/10000 [1:59:03<9:56:37,  4.30s/it]
 17%|█▋        | 1678/10000 [1:59:07<10:02:58,  4.35s/it]
 17%|█▋        | 1679/10000 [1:59:11<9:55:50,  4.30s/it] 
 17%|█▋        | 1680/10000 [1:59:15<9:47:34,  4.24s/it]
                                                        

 17%|█▋        | 1680/10000 [1:59:15<9:47:34,  4.24s/it]
 17%|█▋        | 1681/10000 [1:59:20<9:57:22,  4.31s/it]
 17%|█▋        | 1682/10000 [1:59:24<9:51:49,  4.27s/it]
 17%|█▋        | 1683/10000 [1:59:28<9:46:39,  4.23s/it]
 17%|█▋        | 1684/10000 [1:59:32<9:45:19,  4.22s/it]
 17%|█▋        | 1685/10000 [1:59:38<10:24:03,  4.50s/it]
 17%|█▋        | 1686/10000 [1:59:42<10:10:28,  4.41s/it]
 17%|█▋        | 1687/10000 [1:59:46<10:03:43,  4.36s/it]
 17%|█▋        | 1688/10000 [1:59:50<9:56:36,  4.31s/it] 
 17%|█▋        | 1689/10000 [1:59:54<9:51:17,  4.27s/it]
 17%|█▋        | 1690/10000 [1:59:59<10:04:40,  4.37s/it]
 17%|█▋        | 1691/10000 [2:00:03<9:56:52,  4.31s/it] 
 17%|█▋        | 1692/10000 [2:00:07<9:50:25,  4.26s/it]
 17%|█▋        | 1693/10000 [2:00:12<9:55:31,  4.30s/it]
 17%|█▋        | 1694/10000 [2:00:16<9:50:22,  4.26s/it]
 17%|█▋        | 1695/10000 [2:00:20<9:46:32,  4.24s/it]
 17%|█▋        | 1696/10000 [2:00:25<9:53:29,  4.29s/it]
 17%|█▋        | 1697/10000 [2:00:29<9:48:17,  4.25s/it]
 17%|█▋        | 1698/10000 [2:00:33<9:44:05,  4.22s/it]
 17%|█▋        | 1699/10000 [2:00:37<9:43:42,  4.22s/it]
 17%|█▋        | 1700/10000 [2:00:41<9:40:15,  4.19s/it]
                                                        

 17%|█▋        | 1700/10000 [2:00:41<9:40:15,  4.19s/it]
 17%|█▋        | 1701/10000 [2:00:45<9:38:06,  4.18s/it]
 17%|█▋        | 1702/10000 [2:00:50<9:49:17,  4.26s/it]
 17%|█▋        | 1703/10000 [2:00:54<9:45:03,  4.23s/it]
 17%|█▋        | 1704/10000 [2:00:58<9:41:48,  4.21s/it]
 17%|█▋        | 1705/10000 [2:01:02<9:42:47,  4.22s/it]
 17%|█▋        | 1706/10000 [2:01:07<9:44:39,  4.23s/it]
 17%|█▋        | 1707/10000 [2:01:11<9:42:24,  4.21s/it]
 17%|█▋        | 1708/10000 [2:01:15<9:53:01,  4.29s/it]
 17%|█▋        | 1709/10000 [2:01:19<9:39:16,  4.19s/it]
 17%|█▋        | 1710/10000 [2:01:23<9:41:04,  4.21s/it]
 17%|█▋        | 1711/10000 [2:01:28<9:51:51,  4.28s/it]
 17%|█▋        | 1712/10000 [2:01:32<9:49:53,  4.27s/it]
 17%|█▋        | 1713/10000 [2:01:36<9:46:03,  4.24s/it]
 17%|█▋        | 1714/10000 [2:01:40<9:41:36,  4.21s/it]
 17%|█▋        | 1715/10000 [2:01:45<10:08:31,  4.41s/it]
 17%|█▋        | 1716/10000 [2:01:49<9:58:03,  4.33s/it] 
 17%|█▋        | 1717/10000 [2:01:54<10:16:26,  4.47s/it]
 17%|█▋        | 1718/10000 [2:01:59<10:13:11,  4.44s/it]
 17%|█▋        | 1719/10000 [2:02:03<10:01:52,  4.36s/it]
 17%|█▋        | 1720/10000 [2:02:07<9:53:52,  4.30s/it] 
                                                        

 17%|█▋        | 1720/10000 [2:02:07<9:53:52,  4.30s/it]
 17%|█▋        | 1721/10000 [2:02:11<10:00:10,  4.35s/it]
 17%|█▋        | 1722/10000 [2:02:16<9:50:52,  4.28s/it] 
 17%|█▋        | 1723/10000 [2:02:20<9:46:06,  4.25s/it]
 17%|█▋        | 1724/10000 [2:02:24<9:45:35,  4.25s/it]
 17%|█▋        | 1725/10000 [2:02:28<9:43:05,  4.23s/it]
 17%|█▋        | 1726/10000 [2:02:32<9:43:38,  4.23s/it]
 17%|█▋        | 1727/10000 [2:02:37<9:53:17,  4.30s/it]
 17%|█▋        | 1728/10000 [2:02:41<9:48:10,  4.27s/it]
 17%|█▋        | 1729/10000 [2:02:45<9:34:46,  4.17s/it]
 17%|█▋        | 1730/10000 [2:02:49<9:36:14,  4.18s/it]
 17%|█▋        | 1731/10000 [2:02:54<9:41:45,  4.22s/it]
 17%|█▋        | 1732/10000 [2:02:58<9:37:55,  4.19s/it]
 17%|█▋        | 1733/10000 [2:03:02<9:48:52,  4.27s/it]
 17%|█▋        | 1734/10000 [2:03:06<9:45:48,  4.25s/it]
 17%|█▋        | 1735/10000 [2:03:10<9:33:02,  4.16s/it]
 17%|█▋        | 1736/10000 [2:03:15<9:44:17,  4.24s/it]
 17%|█▋        | 1737/10000 [2:03:19<9:40:47,  4.22s/it]
 17%|█▋        | 1738/10000 [2:03:23<9:37:30,  4.19s/it]
 17%|█▋        | 1739/10000 [2:03:28<9:52:34,  4.30s/it]
 17%|█▋        | 1740/10000 [2:03:32<10:12:13,  4.45s/it]
                                                         

 17%|█▋        | 1740/10000 [2:03:32<10:12:13,  4.45s/it]
 17%|█▋        | 1741/10000 [2:03:36<10:00:09,  4.36s/it]
 17%|█▋        | 1742/10000 [2:03:41<9:54:36,  4.32s/it] 
 17%|█▋        | 1743/10000 [2:03:45<10:02:10,  4.38s/it]
 17%|█▋        | 1744/10000 [2:03:49<9:53:22,  4.31s/it] 
 17%|█▋        | 1745/10000 [2:03:54<9:59:25,  4.36s/it]
 17%|█▋        | 1746/10000 [2:03:58<9:49:19,  4.28s/it]
 17%|█▋        | 1747/10000 [2:04:02<9:44:36,  4.25s/it]
 17%|█▋        | 1748/10000 [2:04:06<9:40:52,  4.22s/it]
 17%|█▋        | 1749/10000 [2:04:11<10:07:15,  4.42s/it]
 18%|█▊        | 1750/10000 [2:04:15<9:57:37,  4.35s/it] 
 18%|█▊        | 1751/10000 [2:04:20<9:49:49,  4.29s/it]
 18%|█▊        | 1752/10000 [2:04:24<9:56:36,  4.34s/it]
 18%|█▊        | 1753/10000 [2:04:28<9:48:51,  4.28s/it]
 18%|█▊        | 1754/10000 [2:04:32<9:43:27,  4.25s/it]
 18%|█▊        | 1755/10000 [2:04:37<10:09:33,  4.44s/it]
 18%|█▊        | 1756/10000 [2:04:41<10:02:22,  4.38s/it]
 18%|█▊        | 1757/10000 [2:04:46<9:53:04,  4.32s/it] 
 18%|█▊        | 1758/10000 [2:04:50<10:10:54,  4.45s/it]
 18%|█▊        | 1759/10000 [2:04:55<10:04:09,  4.40s/it]
 18%|█▊        | 1760/10000 [2:04:59<9:58:03,  4.35s/it] 
                                                        

 18%|█▊        | 1760/10000 [2:04:59<9:58:03,  4.35s/it]
 18%|█▊        | 1761/10000 [2:05:03<10:02:03,  4.38s/it]
 18%|█▊        | 1762/10000 [2:05:07<9:52:41,  4.32s/it] 
 18%|█▊        | 1763/10000 [2:05:12<9:45:52,  4.27s/it]
 18%|█▊        | 1764/10000 [2:05:16<9:58:27,  4.36s/it]
 18%|█▊        | 1765/10000 [2:05:21<10:03:21,  4.40s/it]
 18%|█▊        | 1766/10000 [2:05:25<9:53:18,  4.32s/it] 
 18%|█▊        | 1767/10000 [2:05:30<10:18:17,  4.51s/it]
 18%|█▊        | 1768/10000 [2:05:34<10:03:53,  4.40s/it]
 18%|█▊        | 1769/10000 [2:05:38<9:53:41,  4.33s/it] 
 18%|█▊        | 1770/10000 [2:05:43<10:00:04,  4.37s/it]
 18%|█▊        | 1771/10000 [2:05:47<9:50:41,  4.31s/it] 
 18%|█▊        | 1772/10000 [2:05:51<9:34:57,  4.19s/it]
 18%|█▊        | 1773/10000 [2:05:55<9:45:15,  4.27s/it]
 18%|█▊        | 1774/10000 [2:05:59<9:49:47,  4.30s/it]
 18%|█▊        | 1775/10000 [2:06:04<9:51:44,  4.32s/it]
 18%|█▊        | 1776/10000 [2:06:08<9:59:59,  4.38s/it]
 18%|█▊        | 1777/10000 [2:06:12<9:51:03,  4.31s/it]
 18%|█▊        | 1778/10000 [2:06:17<9:44:20,  4.26s/it]
 18%|█▊        | 1779/10000 [2:06:21<9:51:12,  4.31s/it]
 18%|█▊        | 1780/10000 [2:06:25<9:44:23,  4.27s/it]
                                                        

 18%|█▊        | 1780/10000 [2:06:25<9:44:23,  4.27s/it]
 18%|█▊        | 1781/10000 [2:06:29<9:39:53,  4.23s/it]
 18%|█▊        | 1782/10000 [2:06:34<9:44:25,  4.27s/it]
 18%|█▊        | 1783/10000 [2:06:38<9:31:36,  4.17s/it]
 18%|█▊        | 1784/10000 [2:06:42<9:25:45,  4.13s/it]
 18%|█▊        | 1785/10000 [2:06:46<9:34:54,  4.20s/it]
 18%|█▊        | 1786/10000 [2:06:50<9:27:49,  4.15s/it]
 18%|█▊        | 1787/10000 [2:06:54<9:13:49,  4.05s/it]
 18%|█▊        | 1788/10000 [2:06:58<9:25:47,  4.13s/it]
 18%|█▊        | 1789/10000 [2:07:02<9:18:29,  4.08s/it]
 18%|█▊        | 1790/10000 [2:07:06<9:16:37,  4.07s/it]
 18%|█▊        | 1791/10000 [2:07:10<9:23:39,  4.12s/it]
 18%|█▊        | 1792/10000 [2:07:14<9:11:33,  4.03s/it]
 18%|█▊        | 1793/10000 [2:07:18<9:05:23,  3.99s/it]
 18%|█▊        | 1794/10000 [2:07:22<9:09:38,  4.02s/it]
 18%|█▊        | 1795/10000 [2:07:27<9:26:59,  4.15s/it]
 18%|█▊        | 1796/10000 [2:07:31<9:23:52,  4.12s/it]
 18%|█▊        | 1797/10000 [2:07:35<9:26:04,  4.14s/it]
 18%|█▊        | 1798/10000 [2:07:40<9:49:52,  4.32s/it]
 18%|█▊        | 1799/10000 [2:07:44<9:32:45,  4.19s/it]
 18%|█▊        | 1800/10000 [2:07:48<9:30:46,  4.18s/it]
                                                        

 18%|█▊        | 1800/10000 [2:07:48<9:30:46,  4.18s/it]
 18%|█▊        | 1801/10000 [2:07:52<9:33:14,  4.19s/it]
 18%|█▊        | 1802/10000 [2:07:56<9:31:20,  4.18s/it]
 18%|█▊        | 1803/10000 [2:08:00<9:21:00,  4.11s/it]
 18%|█▊        | 1804/10000 [2:08:05<9:34:21,  4.20s/it]
 18%|█▊        | 1805/10000 [2:08:09<9:40:20,  4.25s/it]
 18%|█▊        | 1806/10000 [2:08:13<9:37:56,  4.23s/it]
 18%|█▊        | 1807/10000 [2:08:17<9:42:42,  4.27s/it]
 18%|█▊        | 1808/10000 [2:08:22<9:37:45,  4.23s/it]
 18%|█▊        | 1809/10000 [2:08:26<9:32:32,  4.19s/it]
 18%|█▊        | 1810/10000 [2:08:30<9:43:06,  4.27s/it]
 18%|█▊        | 1811/10000 [2:08:34<9:44:12,  4.28s/it]
 18%|█▊        | 1812/10000 [2:08:39<9:39:52,  4.25s/it]
 18%|█▊        | 1813/10000 [2:08:43<9:36:09,  4.22s/it]
 18%|█▊        | 1814/10000 [2:08:47<9:37:25,  4.23s/it]
 18%|█▊        | 1815/10000 [2:08:51<9:33:42,  4.21s/it]
 18%|█▊        | 1816/10000 [2:08:55<9:31:42,  4.19s/it]
 18%|█▊        | 1817/10000 [2:09:00<9:34:05,  4.21s/it]
 18%|█▊        | 1818/10000 [2:09:04<9:33:10,  4.20s/it]
 18%|█▊        | 1819/10000 [2:09:08<9:33:53,  4.21s/it]
 18%|█▊        | 1820/10000 [2:09:13<9:47:32,  4.31s/it]
                                                        
{'loss': '0.1313', 'grad_norm': '20.19', 'learning_rate': '4.181e-06', 'num_tokens': '4.87e+05', 'completions/mean_length': '4.081', 'completions/min_length': '3.5', 'completions/max_length': '4.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.081', 'completions/min_terminated_length': '3.5', 'completions/max_terminated_length': '4.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4877', 'rewards/reward_semantic/std': '0.1274', 'rewards/reward_length/mean': '-0.03137', 'rewards/reward_length/std': '0.01074', 'reward': '1.145', 'reward_std': '0.03797', 'frac_reward_zero_std': '0.6', 'kl': '3.167', 'entropy': '0.1627', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.207', 'epoch': '0.269'}
{'loss': '0.06818', 'grad_norm': '15.93', 'learning_rate': '4.171e-06', 'num_tokens': '4.929e+05', 'completions/mean_length': '5.25', 'completions/min_length': '4.55', 'completions/max_length': '5.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.25', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2679', 'rewards/reward_semantic/std': '0.01463', 'rewards/reward_length/mean': '-0.02954', 'rewards/reward_length/std': '0.007468', 'reward': '1.079', 'reward_std': '0.004588', 'frac_reward_zero_std': '0.65', 'kl': '2.333', 'entropy': '0.1815', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.298', 'epoch': '0.2723'}
{'loss': '0.117', 'grad_norm': '0.0003886', 'learning_rate': '4.161e-06', 'num_tokens': '4.987e+05', 'completions/mean_length': '4.869', 'completions/min_length': '4.65', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.869', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3588', 'rewards/reward_semantic/std': '0.005556', 'rewards/reward_length/mean': '-0.02783', 'rewards/reward_length/std': '0.004125', 'reward': '1.106', 'reward_std': '0.001697', 'frac_reward_zero_std': '0.75', 'kl': '2.717', 'entropy': '0.1599', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.247', 'epoch': '0.2756'}
{'loss': '0.1244', 'grad_norm': '0.0002187', 'learning_rate': '4.151e-06', 'num_tokens': '5.045e+05', 'completions/mean_length': '4.737', 'completions/min_length': '4.6', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.737', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3271', 'rewards/reward_semantic/std': '0.04686', 'rewards/reward_length/mean': '-0.0291', 'rewards/reward_length/std': '0.00459', 'reward': '1.097', 'reward_std': '0.01411', 'frac_reward_zero_std': '0.75', 'kl': '2.94', 'entropy': '0.1218', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.264', 'epoch': '0.2789'}
{'loss': '0.1191', 'grad_norm': '0.02578', 'learning_rate': '4.141e-06', 'num_tokens': '5.106e+05', 'completions/mean_length': '4.925', 'completions/min_length': '4.65', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.925', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.306', 'rewards/reward_semantic/std': '0.03573', 'rewards/reward_length/mean': '-0.02395', 'rewards/reward_length/std': '0.004133', 'reward': '1.091', 'reward_std': '0.01078', 'frac_reward_zero_std': '0.75', 'kl': '2.791', 'entropy': '0.1285', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.27', 'epoch': '0.2822'}
{'loss': '0.1213', 'grad_norm': '5.566', 'learning_rate': '4.13e-06', 'num_tokens': '5.162e+05', 'completions/mean_length': '4.537', 'completions/min_length': '4.05', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.537', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4734', 'rewards/reward_semantic/std': '0.06155', 'rewards/reward_length/mean': '-0.0228', 'rewards/reward_length/std': '0.003676', 'reward': '1.141', 'reward_std': '0.01834', 'frac_reward_zero_std': '0.65', 'kl': '3.031', 'entropy': '0.1562', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.247', 'epoch': '0.2854'}
{'loss': '0.1109', 'grad_norm': '18.96', 'learning_rate': '4.121e-06', 'num_tokens': '5.222e+05', 'completions/mean_length': '5.025', 'completions/min_length': '4', 'completions/max_length': '5.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.025', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.292', 'rewards/reward_semantic/std': '0.1266', 'rewards/reward_length/mean': '-0.03557', 'rewards/reward_length/std': '0.007819', 'reward': '1.086', 'reward_std': '0.0379', 'frac_reward_zero_std': '0.55', 'kl': '2.776', 'entropy': '0.2333', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.306', 'epoch': '0.2887'}
{'loss': '0.1403', 'grad_norm': '0.05521', 'learning_rate': '4.111e-06', 'num_tokens': '5.282e+05', 'completions/mean_length': '4.763', 'completions/min_length': '4.1', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.763', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1669', 'rewards/reward_semantic/std': '0.05105', 'rewards/reward_length/mean': '-0.02563', 'rewards/reward_length/std': '0.01025', 'reward': '1.049', 'reward_std': '0.01555', 'frac_reward_zero_std': '0.5', 'kl': '2.553', 'entropy': '0.2372', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.299', 'epoch': '0.292'}
{'loss': '0.06758', 'grad_norm': '0.0003459', 'learning_rate': '4.101e-06', 'num_tokens': '5.341e+05', 'completions/mean_length': '3.931', 'completions/min_length': '3.65', 'completions/max_length': '4.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '3.931', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '4.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4813', 'rewards/reward_semantic/std': '0.01768', 'rewards/reward_length/mean': '-0.03657', 'rewards/reward_length/std': '0.003622', 'reward': '1.143', 'reward_std': '0.005391', 'frac_reward_zero_std': '0.6', 'kl': '1.582', 'entropy': '0.1905', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.107', 'epoch': '0.2953'}

 18%|█▊        | 1820/10000 [2:09:13<9:47:32,  4.31s/it]
 18%|█▊        | 1821/10000 [2:09:17<9:44:03,  4.28s/it]
 18%|█▊        | 1822/10000 [2:09:21<9:39:29,  4.25s/it]
 18%|█▊        | 1823/10000 [2:09:25<9:36:53,  4.23s/it]
 18%|█▊        | 1824/10000 [2:09:30<9:47:29,  4.31s/it]
 18%|█▊        | 1825/10000 [2:09:34<9:41:10,  4.27s/it]
 18%|█▊        | 1826/10000 [2:09:38<9:27:29,  4.17s/it]
 18%|█▊        | 1827/10000 [2:09:42<9:46:32,  4.31s/it]
 18%|█▊        | 1828/10000 [2:09:46<9:39:42,  4.26s/it]
 18%|█▊        | 1829/10000 [2:09:51<9:34:46,  4.22s/it]
 18%|█▊        | 1830/10000 [2:09:55<9:55:07,  4.37s/it]
 18%|█▊        | 1831/10000 [2:09:59<9:37:40,  4.24s/it]
 18%|█▊        | 1832/10000 [2:10:03<9:30:48,  4.19s/it]
 18%|█▊        | 1833/10000 [2:10:08<9:34:29,  4.22s/it]
 18%|█▊        | 1834/10000 [2:10:12<9:24:09,  4.15s/it]
 18%|█▊        | 1835/10000 [2:10:16<9:24:22,  4.15s/it]
 18%|█▊        | 1836/10000 [2:10:20<9:47:28,  4.32s/it]
 18%|█▊        | 1837/10000 [2:10:25<9:47:44,  4.32s/it]
 18%|█▊        | 1838/10000 [2:10:29<9:43:35,  4.29s/it]
 18%|█▊        | 1839/10000 [2:10:33<9:49:21,  4.33s/it]
 18%|█▊        | 1840/10000 [2:10:38<9:42:00,  4.28s/it]
                                                        

 18%|█▊        | 1840/10000 [2:10:38<9:42:00,  4.28s/it]
 18%|█▊        | 1841/10000 [2:10:42<9:43:52,  4.29s/it]
 18%|█▊        | 1842/10000 [2:10:46<9:50:01,  4.34s/it]
 18%|█▊        | 1843/10000 [2:10:51<9:42:24,  4.28s/it]
 18%|█▊        | 1844/10000 [2:10:55<9:46:38,  4.32s/it]
 18%|█▊        | 1845/10000 [2:10:59<9:52:49,  4.36s/it]
 18%|█▊        | 1846/10000 [2:11:03<9:38:15,  4.26s/it]
 18%|█▊        | 1847/10000 [2:11:07<9:25:18,  4.16s/it]
 18%|█▊        | 1848/10000 [2:11:12<9:56:00,  4.39s/it]
 18%|█▊        | 1849/10000 [2:11:16<9:47:11,  4.32s/it]
 18%|█▊        | 1850/10000 [2:11:21<9:40:43,  4.28s/it]
 19%|█▊        | 1851/10000 [2:11:25<9:48:39,  4.33s/it]
 19%|█▊        | 1852/10000 [2:11:30<9:58:17,  4.41s/it]
 19%|█▊        | 1853/10000 [2:11:34<10:06:29,  4.47s/it]
 19%|█▊        | 1854/10000 [2:11:38<9:55:57,  4.39s/it] 
 19%|█▊        | 1855/10000 [2:11:43<9:59:54,  4.42s/it]
 19%|█▊        | 1856/10000 [2:11:47<9:55:57,  4.39s/it]
 19%|█▊        | 1857/10000 [2:11:51<9:46:12,  4.32s/it]
 19%|█▊        | 1858/10000 [2:11:56<9:54:49,  4.38s/it]
 19%|█▊        | 1859/10000 [2:12:00<9:47:37,  4.33s/it]
 19%|█▊        | 1860/10000 [2:12:05<10:04:27,  4.46s/it]
                                                         

 19%|█▊        | 1860/10000 [2:12:05<10:04:27,  4.46s/it]
 19%|█▊        | 1861/10000 [2:12:09<10:10:28,  4.50s/it]
 19%|█▊        | 1862/10000 [2:12:14<10:08:18,  4.48s/it]
 19%|█▊        | 1863/10000 [2:12:18<10:10:09,  4.50s/it]
 19%|█▊        | 1864/10000 [2:12:23<10:06:57,  4.48s/it]
 19%|█▊        | 1865/10000 [2:12:27<9:54:48,  4.39s/it] 
 19%|█▊        | 1866/10000 [2:12:31<9:38:49,  4.27s/it]
 19%|█▊        | 1867/10000 [2:12:36<9:45:53,  4.32s/it]
 19%|█▊        | 1868/10000 [2:12:40<9:47:29,  4.33s/it]
 19%|█▊        | 1869/10000 [2:12:44<9:55:59,  4.40s/it]
 19%|█▊        | 1870/10000 [2:12:49<9:59:35,  4.43s/it]
 19%|█▊        | 1871/10000 [2:12:53<9:55:19,  4.39s/it]
 19%|█▊        | 1872/10000 [2:12:58<9:55:31,  4.40s/it]
 19%|█▊        | 1873/10000 [2:13:02<9:49:53,  4.36s/it]
 19%|█▊        | 1874/10000 [2:13:06<9:53:52,  4.39s/it]
 19%|█▉        | 1875/10000 [2:13:10<9:39:37,  4.28s/it]
 19%|█▉        | 1876/10000 [2:13:15<9:33:42,  4.24s/it]
 19%|█▉        | 1877/10000 [2:13:19<9:38:44,  4.27s/it]
 19%|█▉        | 1878/10000 [2:13:23<9:35:34,  4.25s/it]
 19%|█▉        | 1879/10000 [2:13:27<9:33:23,  4.24s/it]
 19%|█▉        | 1880/10000 [2:13:32<9:53:34,  4.39s/it]
                                                        

 19%|█▉        | 1880/10000 [2:13:32<9:53:34,  4.39s/it]
 19%|█▉        | 1881/10000 [2:13:36<9:51:04,  4.37s/it]
 19%|█▉        | 1882/10000 [2:13:41<9:44:46,  4.32s/it]
 19%|█▉        | 1883/10000 [2:13:45<10:02:23,  4.45s/it]
 19%|█▉        | 1884/10000 [2:13:50<9:52:00,  4.38s/it] 
 19%|█▉        | 1885/10000 [2:13:54<9:53:41,  4.39s/it]
 19%|█▉        | 1886/10000 [2:13:58<9:59:21,  4.43s/it]
 19%|█▉        | 1887/10000 [2:14:03<9:59:24,  4.43s/it]
 19%|█▉        | 1888/10000 [2:14:07<9:57:01,  4.42s/it]
 19%|█▉        | 1889/10000 [2:14:12<9:48:52,  4.36s/it]
 19%|█▉        | 1890/10000 [2:14:16<9:45:36,  4.33s/it]
 19%|█▉        | 1891/10000 [2:14:20<9:48:33,  4.35s/it]
 19%|█▉        | 1892/10000 [2:14:24<9:44:05,  4.32s/it]
 19%|█▉        | 1893/10000 [2:14:29<9:42:15,  4.31s/it]
 19%|█▉        | 1894/10000 [2:14:33<9:37:58,  4.28s/it]
 19%|█▉        | 1895/10000 [2:14:37<9:37:37,  4.28s/it]
 19%|█▉        | 1896/10000 [2:14:42<9:41:58,  4.31s/it]
 19%|█▉        | 1897/10000 [2:14:46<9:37:04,  4.27s/it]
 19%|█▉        | 1898/10000 [2:14:50<9:46:37,  4.34s/it]
 19%|█▉        | 1899/10000 [2:14:54<9:40:50,  4.30s/it]
 19%|█▉        | 1900/10000 [2:14:59<9:44:55,  4.33s/it]
                                                        

 19%|█▉        | 1900/10000 [2:14:59<9:44:55,  4.33s/it]
 19%|█▉        | 1901/10000 [2:15:04<10:04:41,  4.48s/it]
 19%|█▉        | 1902/10000 [2:15:08<9:39:27,  4.29s/it] 
 19%|█▉        | 1903/10000 [2:15:12<9:46:58,  4.35s/it]
 19%|█▉        | 1904/10000 [2:15:16<9:31:17,  4.23s/it]
 19%|█▉        | 1905/10000 [2:15:20<9:28:51,  4.22s/it]
 19%|█▉        | 1906/10000 [2:15:24<9:13:49,  4.11s/it]
 19%|█▉        | 1907/10000 [2:15:28<9:21:53,  4.17s/it]
 19%|█▉        | 1908/10000 [2:15:32<9:19:01,  4.15s/it]
 19%|█▉        | 1909/10000 [2:15:37<9:16:48,  4.13s/it]
 19%|█▉        | 1910/10000 [2:15:41<9:27:25,  4.21s/it]
 19%|█▉        | 1911/10000 [2:15:45<9:19:33,  4.15s/it]
 19%|█▉        | 1912/10000 [2:15:49<9:16:43,  4.13s/it]
 19%|█▉        | 1913/10000 [2:15:53<9:27:03,  4.21s/it]
 19%|█▉        | 1914/10000 [2:15:58<9:23:50,  4.18s/it]
 19%|█▉        | 1915/10000 [2:16:02<9:20:05,  4.16s/it]
 19%|█▉        | 1916/10000 [2:16:06<9:29:44,  4.23s/it]
 19%|█▉        | 1917/10000 [2:16:10<9:24:28,  4.19s/it]
 19%|█▉        | 1918/10000 [2:16:14<9:11:27,  4.09s/it]
 19%|█▉        | 1919/10000 [2:16:18<9:23:18,  4.18s/it]
 19%|█▉        | 1920/10000 [2:16:22<9:20:10,  4.16s/it]
                                                        

 19%|█▉        | 1920/10000 [2:16:22<9:20:10,  4.16s/it]
 19%|█▉        | 1921/10000 [2:16:27<9:17:18,  4.14s/it]
 19%|█▉        | 1922/10000 [2:16:31<9:27:18,  4.21s/it]
 19%|█▉        | 1923/10000 [2:16:35<9:21:18,  4.17s/it]
 19%|█▉        | 1924/10000 [2:16:39<9:18:33,  4.15s/it]
 19%|█▉        | 1925/10000 [2:16:43<9:12:40,  4.11s/it]
 19%|█▉        | 1926/10000 [2:16:48<9:23:59,  4.19s/it]
 19%|█▉        | 1927/10000 [2:16:51<9:10:30,  4.09s/it]
 19%|█▉        | 1928/10000 [2:16:55<9:09:42,  4.09s/it]
 19%|█▉        | 1929/10000 [2:17:00<9:22:24,  4.18s/it]
 19%|█▉        | 1930/10000 [2:17:04<9:16:06,  4.13s/it]
 19%|█▉        | 1931/10000 [2:17:08<9:19:06,  4.16s/it]
 19%|█▉        | 1932/10000 [2:17:12<9:28:41,  4.23s/it]
 19%|█▉        | 1933/10000 [2:17:17<9:23:20,  4.19s/it]
 19%|█▉        | 1934/10000 [2:17:21<9:22:38,  4.19s/it]
 19%|█▉        | 1935/10000 [2:17:25<9:36:51,  4.29s/it]
 19%|█▉        | 1936/10000 [2:17:29<9:24:30,  4.20s/it]
 19%|█▉        | 1937/10000 [2:17:33<9:22:00,  4.18s/it]
 19%|█▉        | 1938/10000 [2:17:38<9:17:22,  4.15s/it]
 19%|█▉        | 1939/10000 [2:17:42<9:25:58,  4.21s/it]
 19%|█▉        | 1940/10000 [2:17:46<9:18:52,  4.16s/it]
                                                        

 19%|█▉        | 1940/10000 [2:17:46<9:18:52,  4.16s/it]
 19%|█▉        | 1941/10000 [2:17:50<9:18:17,  4.16s/it]
 19%|█▉        | 1942/10000 [2:17:55<9:36:57,  4.30s/it]
 19%|█▉        | 1943/10000 [2:17:59<9:28:22,  4.23s/it]
 19%|█▉        | 1944/10000 [2:18:03<9:16:46,  4.15s/it]
 19%|█▉        | 1945/10000 [2:18:07<9:31:12,  4.25s/it]
 19%|█▉        | 1946/10000 [2:18:11<9:24:39,  4.21s/it]
 19%|█▉        | 1947/10000 [2:18:15<9:20:20,  4.17s/it]
 19%|█▉        | 1948/10000 [2:18:20<9:37:34,  4.30s/it]
 19%|█▉        | 1949/10000 [2:18:24<9:24:39,  4.21s/it]
 20%|█▉        | 1950/10000 [2:18:28<9:19:31,  4.17s/it]
 20%|█▉        | 1951/10000 [2:18:33<10:00:26,  4.48s/it]
 20%|█▉        | 1952/10000 [2:18:37<9:41:01,  4.33s/it] 
 20%|█▉        | 1953/10000 [2:18:42<9:44:07,  4.36s/it]
 20%|█▉        | 1954/10000 [2:18:46<9:37:19,  4.31s/it]
 20%|█▉        | 1955/10000 [2:18:50<9:45:44,  4.37s/it]
 20%|█▉        | 1956/10000 [2:18:55<9:39:19,  4.32s/it]
 20%|█▉        | 1957/10000 [2:18:59<9:44:43,  4.36s/it]
 20%|█▉        | 1958/10000 [2:19:03<9:41:38,  4.34s/it]
 20%|█▉        | 1959/10000 [2:19:08<9:37:07,  4.31s/it]
 20%|█▉        | 1960/10000 [2:19:12<9:31:19,  4.26s/it]
                                                        

 20%|█▉        | 1960/10000 [2:19:12<9:31:19,  4.26s/it]
 20%|█▉        | 1961/10000 [2:19:16<9:37:25,  4.31s/it]
 20%|█▉        | 1962/10000 [2:19:20<9:39:01,  4.32s/it]
 20%|█▉        | 1963/10000 [2:19:25<9:34:44,  4.29s/it]
 20%|█▉        | 1964/10000 [2:19:29<9:33:17,  4.28s/it]
 20%|█▉        | 1965/10000 [2:19:33<9:32:40,  4.28s/it]
 20%|█▉        | 1966/10000 [2:19:37<9:29:29,  4.25s/it]
 20%|█▉        | 1967/10000 [2:19:42<9:59:58,  4.48s/it]
 20%|█▉        | 1968/10000 [2:19:47<9:59:03,  4.48s/it]
 20%|█▉        | 1969/10000 [2:19:51<9:43:07,  4.36s/it]
 20%|█▉        | 1970/10000 [2:19:55<9:39:45,  4.33s/it]
 20%|█▉        | 1971/10000 [2:19:59<9:28:35,  4.25s/it]
 20%|█▉        | 1972/10000 [2:20:04<9:28:06,  4.25s/it]
 20%|█▉        | 1973/10000 [2:20:08<9:33:30,  4.29s/it]
 20%|█▉        | 1974/10000 [2:20:12<9:23:47,  4.21s/it]
 20%|█▉        | 1975/10000 [2:20:16<9:18:22,  4.17s/it]
 20%|█▉        | 1976/10000 [2:20:20<9:17:39,  4.17s/it]
 20%|█▉        | 1977/10000 [2:20:24<9:17:07,  4.17s/it]
 20%|█▉        | 1978/10000 [2:20:29<9:15:22,  4.15s/it]
 20%|█▉        | 1979/10000 [2:20:33<9:26:27,  4.24s/it]
 20%|█▉        | 1980/10000 [2:20:37<9:30:48,  4.27s/it]
                                                        

 20%|█▉        | 1980/10000 [2:20:37<9:30:48,  4.27s/it]
 20%|█▉        | 1981/10000 [2:20:42<9:31:53,  4.28s/it]
 20%|█▉        | 1982/10000 [2:20:46<9:32:03,  4.28s/it]
 20%|█▉        | 1983/10000 [2:20:50<9:44:56,  4.38s/it]
 20%|█▉        | 1984/10000 [2:20:55<9:41:31,  4.35s/it]
 20%|█▉        | 1985/10000 [2:20:59<9:38:56,  4.33s/it]
 20%|█▉        | 1986/10000 [2:21:04<9:49:47,  4.42s/it]
 20%|█▉        | 1987/10000 [2:21:08<9:41:23,  4.35s/it]
 20%|█▉        | 1988/10000 [2:21:12<9:31:28,  4.28s/it]
 20%|█▉        | 1989/10000 [2:21:17<9:43:11,  4.37s/it]
 20%|█▉        | 1990/10000 [2:21:21<9:32:16,  4.29s/it]
 20%|█▉        | 1991/10000 [2:21:25<9:23:10,  4.22s/it]
 20%|█▉        | 1992/10000 [2:21:29<9:40:26,  4.35s/it]
 20%|█▉        | 1993/10000 [2:21:33<9:29:35,  4.27s/it]
 20%|█▉        | 1994/10000 [2:21:38<9:30:24,  4.27s/it]
 20%|█▉        | 1995/10000 [2:21:42<9:34:31,  4.31s/it]
 20%|█▉        | 1996/10000 [2:21:47<9:42:23,  4.37s/it]
 20%|█▉        | 1997/10000 [2:21:51<9:31:12,  4.28s/it]
 20%|█▉        | 1998/10000 [2:21:55<9:37:18,  4.33s/it]
 20%|█▉        | 1999/10000 [2:21:59<9:27:03,  4.25s/it]
 20%|██        | 2000/10000 [2:22:03<9:20:30,  4.20s/it]
                                                        
{'loss': '0.06999', 'grad_norm': '5.112', 'learning_rate': '4.091e-06', 'num_tokens': '5.397e+05', 'completions/mean_length': '3.856', 'completions/min_length': '3.55', 'completions/max_length': '4.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '3.856', 'completions/min_terminated_length': '3.55', 'completions/max_terminated_length': '4.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4448', 'rewards/reward_semantic/std': '0.09042', 'rewards/reward_length/mean': '-0.03311', 'rewards/reward_length/std': '0.006789', 'reward': '1.132', 'reward_std': '0.02712', 'frac_reward_zero_std': '0.6', 'kl': '1.56', 'entropy': '0.2025', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.218', 'epoch': '0.2986'}
{'loss': '0.05503', 'grad_norm': '0.2279', 'learning_rate': '4.081e-06', 'num_tokens': '5.459e+05', 'completions/mean_length': '4.194', 'completions/min_length': '4.05', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.194', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4147', 'rewards/reward_semantic/std': '0.08443', 'rewards/reward_length/mean': '-0.03445', 'rewards/reward_length/std': '0.004896', 'reward': '1.123', 'reward_std': '0.02534', 'frac_reward_zero_std': '0.7', 'kl': '1.861', 'entropy': '0.2237', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.234', 'epoch': '0.3018'}
{'loss': '0.08491', 'grad_norm': '2.732', 'learning_rate': '4.071e-06', 'num_tokens': '5.52e+05', 'completions/mean_length': '4.838', 'completions/min_length': '4.2', 'completions/max_length': '5.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.838', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '5.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1968', 'rewards/reward_semantic/std': '0.07442', 'rewards/reward_length/mean': '-0.03955', 'rewards/reward_length/std': '0.01328', 'reward': '1.057', 'reward_std': '0.02259', 'frac_reward_zero_std': '0.3', 'kl': '1.549', 'entropy': '0.3408', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.345', 'epoch': '0.3051'}
{'loss': '0.1465', 'grad_norm': '17.78', 'learning_rate': '4.061e-06', 'num_tokens': '5.579e+05', 'completions/mean_length': '4.594', 'completions/min_length': '3.95', 'completions/max_length': '5.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.594', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '5.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3247', 'rewards/reward_semantic/std': '0.1043', 'rewards/reward_length/mean': '-0.03787', 'rewards/reward_length/std': '0.01219', 'reward': '1.096', 'reward_std': '0.03156', 'frac_reward_zero_std': '0.4', 'kl': '1.461', 'entropy': '0.3796', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.337', 'epoch': '0.3084'}
{'loss': '0.09555', 'grad_norm': '0.008211', 'learning_rate': '4.051e-06', 'num_tokens': '5.641e+05', 'completions/mean_length': '4.612', 'completions/min_length': '4.25', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.612', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3463', 'rewards/reward_semantic/std': '0.09585', 'rewards/reward_length/mean': '-0.03794', 'rewards/reward_length/std': '0.01097', 'reward': '1.102', 'reward_std': '0.02883', 'frac_reward_zero_std': '0.4', 'kl': '1.973', 'entropy': '0.2767', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.323', 'epoch': '0.3117'}
{'loss': '0.09831', 'grad_norm': '0.01563', 'learning_rate': '4.041e-06', 'num_tokens': '5.697e+05', 'completions/mean_length': '4.188', 'completions/min_length': '3.7', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.188', 'completions/min_terminated_length': '3.7', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3638', 'rewards/reward_semantic/std': '0.05265', 'rewards/reward_length/mean': '-0.03042', 'rewards/reward_length/std': '0.005515', 'reward': '1.108', 'reward_std': '0.01587', 'frac_reward_zero_std': '0.6', 'kl': '2.137', 'entropy': '0.2165', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.162', 'epoch': '0.315'}
{'loss': '0.1195', 'grad_norm': '0.03603', 'learning_rate': '4.031e-06', 'num_tokens': '5.754e+05', 'completions/mean_length': '4.344', 'completions/min_length': '4.05', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.344', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.258', 'rewards/reward_semantic/std': '0.1146', 'rewards/reward_length/mean': '-0.03137', 'rewards/reward_length/std': '0.008505', 'reward': '1.076', 'reward_std': '0.03432', 'frac_reward_zero_std': '0.45', 'kl': '2.017', 'entropy': '0.2044', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.153', 'epoch': '0.3182'}
{'loss': '0.05202', 'grad_norm': '0.002011', 'learning_rate': '4.021e-06', 'num_tokens': '5.81e+05', 'completions/mean_length': '4.231', 'completions/min_length': '3.65', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.231', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3591', 'rewards/reward_semantic/std': '0.06713', 'rewards/reward_length/mean': '-0.03774', 'rewards/reward_length/std': '0.01086', 'reward': '1.106', 'reward_std': '0.02004', 'frac_reward_zero_std': '0.5', 'kl': '1.42', 'entropy': '0.3122', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.272', 'epoch': '0.3215'}
{'loss': '0.05818', 'grad_norm': '18.04', 'learning_rate': '4.011e-06', 'num_tokens': '5.868e+05', 'completions/mean_length': '4.594', 'completions/min_length': '3.9', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.594', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4009', 'rewards/reward_semantic/std': '0.07668', 'rewards/reward_length/mean': '-0.0394', 'rewards/reward_length/std': '0.01083', 'reward': '1.118', 'reward_std': '0.02296', 'frac_reward_zero_std': '0.4', 'kl': '1.547', 'entropy': '0.2691', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.259', 'epoch': '0.3248'}

 20%|██        | 2000/10000 [2:22:03<9:20:30,  4.20s/it]
 20%|██        | 2001/10000 [2:22:09<10:15:20,  4.62s/it]
 20%|██        | 2002/10000 [2:22:13<9:52:54,  4.45s/it] 
 20%|██        | 2003/10000 [2:22:17<9:27:21,  4.26s/it]
 20%|██        | 2004/10000 [2:22:21<9:09:35,  4.12s/it]
 20%|██        | 2005/10000 [2:22:25<9:28:14,  4.26s/it]
 20%|██        | 2006/10000 [2:22:29<9:20:28,  4.21s/it]
 20%|██        | 2007/10000 [2:22:34<9:25:59,  4.25s/it]
 20%|██        | 2008/10000 [2:22:38<9:18:10,  4.19s/it]
 20%|██        | 2009/10000 [2:22:42<9:09:39,  4.13s/it]
 20%|██        | 2010/10000 [2:22:46<9:11:33,  4.14s/it]
 20%|██        | 2011/10000 [2:22:50<9:23:18,  4.23s/it]
 20%|██        | 2012/10000 [2:22:54<9:18:29,  4.20s/it]
 20%|██        | 2013/10000 [2:22:58<9:13:56,  4.16s/it]
 20%|██        | 2014/10000 [2:23:03<9:39:24,  4.35s/it]
 20%|██        | 2015/10000 [2:23:07<9:22:10,  4.22s/it]
 20%|██        | 2016/10000 [2:23:11<9:12:57,  4.16s/it]
 20%|██        | 2017/10000 [2:23:16<9:38:58,  4.35s/it]
 20%|██        | 2018/10000 [2:23:20<9:28:15,  4.27s/it]
 20%|██        | 2019/10000 [2:23:24<9:32:18,  4.30s/it]
 20%|██        | 2020/10000 [2:23:28<9:22:22,  4.23s/it]
                                                        

 20%|██        | 2020/10000 [2:23:28<9:22:22,  4.23s/it]
 20%|██        | 2021/10000 [2:23:33<9:16:45,  4.19s/it]
 20%|██        | 2022/10000 [2:23:37<9:24:04,  4.24s/it]
 20%|██        | 2023/10000 [2:23:41<9:16:24,  4.19s/it]
 20%|██        | 2024/10000 [2:23:45<9:12:10,  4.15s/it]
 20%|██        | 2025/10000 [2:23:49<9:19:46,  4.21s/it]
 20%|██        | 2026/10000 [2:23:53<9:07:41,  4.12s/it]
 20%|██        | 2027/10000 [2:23:58<9:12:51,  4.16s/it]
 20%|██        | 2028/10000 [2:24:02<9:21:31,  4.23s/it]
 20%|██        | 2029/10000 [2:24:06<9:16:53,  4.19s/it]
 20%|██        | 2030/10000 [2:24:10<9:17:21,  4.20s/it]
 20%|██        | 2031/10000 [2:24:15<9:23:28,  4.24s/it]
 20%|██        | 2032/10000 [2:24:19<9:16:30,  4.19s/it]
 20%|██        | 2033/10000 [2:24:23<9:09:09,  4.14s/it]
 20%|██        | 2034/10000 [2:24:27<9:17:49,  4.20s/it]
 20%|██        | 2035/10000 [2:24:32<9:37:29,  4.35s/it]
 20%|██        | 2036/10000 [2:24:36<9:25:14,  4.26s/it]
 20%|██        | 2037/10000 [2:24:40<9:42:38,  4.39s/it]
 20%|██        | 2038/10000 [2:24:45<9:29:03,  4.29s/it]
 20%|██        | 2039/10000 [2:24:49<9:20:02,  4.22s/it]
 20%|██        | 2040/10000 [2:24:53<9:25:20,  4.26s/it]
                                                        

 20%|██        | 2040/10000 [2:24:53<9:25:20,  4.26s/it]
 20%|██        | 2041/10000 [2:24:57<9:17:20,  4.20s/it]
 20%|██        | 2042/10000 [2:25:01<9:27:54,  4.28s/it]
 20%|██        | 2043/10000 [2:25:05<9:10:30,  4.15s/it]
 20%|██        | 2044/10000 [2:25:09<9:09:09,  4.14s/it]
 20%|██        | 2045/10000 [2:25:14<9:15:58,  4.19s/it]
 20%|██        | 2046/10000 [2:25:18<9:12:10,  4.17s/it]
 20%|██        | 2047/10000 [2:25:22<9:08:33,  4.14s/it]
 20%|██        | 2048/10000 [2:25:26<9:17:54,  4.21s/it]
 20%|██        | 2049/10000 [2:25:30<9:16:11,  4.20s/it]
 20%|██        | 2050/10000 [2:25:35<9:18:56,  4.22s/it]
 21%|██        | 2051/10000 [2:25:39<9:29:14,  4.30s/it]
 21%|██        | 2052/10000 [2:25:43<9:24:21,  4.26s/it]
 21%|██        | 2053/10000 [2:25:47<9:11:44,  4.17s/it]
 21%|██        | 2054/10000 [2:25:52<9:26:16,  4.28s/it]
 21%|██        | 2055/10000 [2:25:56<9:27:03,  4.28s/it]
 21%|██        | 2056/10000 [2:26:00<9:22:56,  4.25s/it]
 21%|██        | 2057/10000 [2:26:05<9:39:25,  4.38s/it]
 21%|██        | 2058/10000 [2:26:09<9:31:45,  4.32s/it]
 21%|██        | 2059/10000 [2:26:14<9:38:14,  4.37s/it]
 21%|██        | 2060/10000 [2:26:18<9:30:45,  4.31s/it]
                                                        

 21%|██        | 2060/10000 [2:26:18<9:30:45,  4.31s/it]
 21%|██        | 2061/10000 [2:26:22<9:26:10,  4.28s/it]
 21%|██        | 2062/10000 [2:26:27<9:36:31,  4.36s/it]
 21%|██        | 2063/10000 [2:26:31<9:33:42,  4.34s/it]
 21%|██        | 2064/10000 [2:26:35<9:23:45,  4.26s/it]
 21%|██        | 2065/10000 [2:26:39<9:22:51,  4.26s/it]
 21%|██        | 2066/10000 [2:26:43<9:19:24,  4.23s/it]
 21%|██        | 2067/10000 [2:26:48<9:17:18,  4.22s/it]
 21%|██        | 2068/10000 [2:26:52<9:26:30,  4.29s/it]
 21%|██        | 2069/10000 [2:26:56<9:23:21,  4.26s/it]
 21%|██        | 2070/10000 [2:27:00<9:21:04,  4.25s/it]
 21%|██        | 2071/10000 [2:27:05<9:33:11,  4.34s/it]
 21%|██        | 2072/10000 [2:27:10<9:39:12,  4.38s/it]
 21%|██        | 2073/10000 [2:27:14<9:29:23,  4.31s/it]
 21%|██        | 2074/10000 [2:27:18<9:36:02,  4.36s/it]
 21%|██        | 2075/10000 [2:27:22<9:19:54,  4.24s/it]
 21%|██        | 2076/10000 [2:27:26<9:08:51,  4.16s/it]
 21%|██        | 2077/10000 [2:27:31<9:21:48,  4.25s/it]
 21%|██        | 2078/10000 [2:27:35<9:19:14,  4.24s/it]
 21%|██        | 2079/10000 [2:27:39<9:39:23,  4.39s/it]
 21%|██        | 2080/10000 [2:27:44<9:31:47,  4.33s/it]
                                                        

 21%|██        | 2080/10000 [2:27:44<9:31:47,  4.33s/it]
 21%|██        | 2081/10000 [2:27:48<9:30:57,  4.33s/it]
 21%|██        | 2082/10000 [2:27:52<9:36:33,  4.37s/it]
 21%|██        | 2083/10000 [2:27:57<9:29:26,  4.32s/it]
 21%|██        | 2084/10000 [2:28:01<9:24:25,  4.28s/it]
 21%|██        | 2085/10000 [2:28:05<9:32:30,  4.34s/it]
 21%|██        | 2086/10000 [2:28:09<9:17:11,  4.22s/it]
 21%|██        | 2087/10000 [2:28:13<9:15:16,  4.21s/it]
 21%|██        | 2088/10000 [2:28:18<9:25:32,  4.29s/it]
 21%|██        | 2089/10000 [2:28:22<9:29:58,  4.32s/it]
 21%|██        | 2090/10000 [2:28:27<9:42:03,  4.42s/it]
 21%|██        | 2091/10000 [2:28:31<9:38:06,  4.39s/it]
 21%|██        | 2092/10000 [2:28:35<9:30:32,  4.33s/it]
 21%|██        | 2093/10000 [2:28:40<9:36:04,  4.37s/it]
 21%|██        | 2094/10000 [2:28:44<9:27:58,  4.31s/it]
 21%|██        | 2095/10000 [2:28:48<9:18:19,  4.24s/it]
 21%|██        | 2096/10000 [2:28:53<9:22:03,  4.27s/it]
 21%|██        | 2097/10000 [2:28:56<9:06:17,  4.15s/it]
 21%|██        | 2098/10000 [2:29:01<9:25:22,  4.29s/it]
 21%|██        | 2099/10000 [2:29:05<9:31:15,  4.34s/it]
 21%|██        | 2100/10000 [2:29:09<9:19:38,  4.25s/it]
                                                        

 21%|██        | 2100/10000 [2:29:09<9:19:38,  4.25s/it]
 21%|██        | 2101/10000 [2:29:14<9:12:14,  4.19s/it]
 21%|██        | 2102/10000 [2:29:18<9:24:07,  4.29s/it]
 21%|██        | 2103/10000 [2:29:23<9:32:23,  4.35s/it]
 21%|██        | 2104/10000 [2:29:27<9:21:35,  4.27s/it]
 21%|██        | 2105/10000 [2:29:31<9:25:41,  4.30s/it]
 21%|██        | 2106/10000 [2:29:35<9:16:36,  4.23s/it]
 21%|██        | 2107/10000 [2:29:39<9:10:28,  4.18s/it]
 21%|██        | 2108/10000 [2:29:43<9:14:45,  4.22s/it]
 21%|██        | 2109/10000 [2:29:48<9:09:13,  4.18s/it]
 21%|██        | 2110/10000 [2:29:52<9:12:44,  4.20s/it]
 21%|██        | 2111/10000 [2:29:56<9:08:16,  4.17s/it]
 21%|██        | 2112/10000 [2:30:00<9:01:28,  4.12s/it]
 21%|██        | 2113/10000 [2:30:04<9:13:24,  4.21s/it]
 21%|██        | 2114/10000 [2:30:08<9:08:31,  4.17s/it]
 21%|██        | 2115/10000 [2:30:12<9:04:43,  4.15s/it]
 21%|██        | 2116/10000 [2:30:17<9:13:35,  4.21s/it]
 21%|██        | 2117/10000 [2:30:21<9:00:59,  4.12s/it]
 21%|██        | 2118/10000 [2:30:25<8:56:22,  4.08s/it]
 21%|██        | 2119/10000 [2:30:29<9:07:53,  4.17s/it]
 21%|██        | 2120/10000 [2:30:33<8:59:10,  4.11s/it]
                                                        

 21%|██        | 2120/10000 [2:30:33<8:59:10,  4.11s/it]
 21%|██        | 2121/10000 [2:30:37<8:58:01,  4.10s/it]
 21%|██        | 2122/10000 [2:30:42<9:09:26,  4.18s/it]
 21%|██        | 2123/10000 [2:30:46<9:06:47,  4.17s/it]
 21%|██        | 2124/10000 [2:30:50<9:05:58,  4.16s/it]
 21%|██▏       | 2125/10000 [2:30:54<9:11:17,  4.20s/it]
 21%|██▏       | 2126/10000 [2:30:58<9:06:41,  4.17s/it]
 21%|██▏       | 2127/10000 [2:31:02<9:03:30,  4.14s/it]
 21%|██▏       | 2128/10000 [2:31:07<9:11:59,  4.21s/it]
 21%|██▏       | 2129/10000 [2:31:11<9:03:34,  4.14s/it]
 21%|██▏       | 2130/10000 [2:31:15<9:16:24,  4.24s/it]
 21%|██▏       | 2131/10000 [2:31:19<9:13:16,  4.22s/it]
 21%|██▏       | 2132/10000 [2:31:23<9:09:14,  4.19s/it]
 21%|██▏       | 2133/10000 [2:31:28<9:16:56,  4.25s/it]
 21%|██▏       | 2134/10000 [2:31:32<9:11:07,  4.20s/it]
 21%|██▏       | 2135/10000 [2:31:36<9:10:05,  4.20s/it]
 21%|██▏       | 2136/10000 [2:31:41<9:23:49,  4.30s/it]
 21%|██▏       | 2137/10000 [2:31:45<9:35:25,  4.39s/it]
 21%|██▏       | 2138/10000 [2:31:50<9:33:13,  4.37s/it]
 21%|██▏       | 2139/10000 [2:31:54<9:35:20,  4.39s/it]
 21%|██▏       | 2140/10000 [2:31:58<9:20:18,  4.28s/it]
                                                        

 21%|██▏       | 2140/10000 [2:31:58<9:20:18,  4.28s/it]
 21%|██▏       | 2141/10000 [2:32:02<9:27:26,  4.33s/it]
 21%|██▏       | 2142/10000 [2:32:07<9:34:28,  4.39s/it]
 21%|██▏       | 2143/10000 [2:32:11<9:28:13,  4.34s/it]
 21%|██▏       | 2144/10000 [2:32:16<9:32:40,  4.37s/it]
 21%|██▏       | 2145/10000 [2:32:20<9:26:56,  4.33s/it]
 21%|██▏       | 2146/10000 [2:32:24<9:20:28,  4.28s/it]
 21%|██▏       | 2147/10000 [2:32:28<9:19:00,  4.27s/it]
 21%|██▏       | 2148/10000 [2:32:33<9:20:07,  4.28s/it]
 21%|██▏       | 2149/10000 [2:32:37<9:15:42,  4.25s/it]
 22%|██▏       | 2150/10000 [2:32:41<9:24:05,  4.31s/it]
 22%|██▏       | 2151/10000 [2:32:45<9:18:23,  4.27s/it]
 22%|██▏       | 2152/10000 [2:32:50<9:13:59,  4.24s/it]
 22%|██▏       | 2153/10000 [2:32:54<9:33:59,  4.39s/it]
 22%|██▏       | 2154/10000 [2:32:59<9:35:22,  4.40s/it]
 22%|██▏       | 2155/10000 [2:33:03<9:34:54,  4.40s/it]
 22%|██▏       | 2156/10000 [2:33:08<9:37:07,  4.41s/it]
 22%|██▏       | 2157/10000 [2:33:12<9:27:08,  4.34s/it]
 22%|██▏       | 2158/10000 [2:33:16<9:31:58,  4.38s/it]
 22%|██▏       | 2159/10000 [2:33:20<9:15:11,  4.25s/it]
 22%|██▏       | 2160/10000 [2:33:24<9:18:42,  4.28s/it]
                                                        

 22%|██▏       | 2160/10000 [2:33:24<9:18:42,  4.28s/it]
 22%|██▏       | 2161/10000 [2:33:29<9:26:08,  4.33s/it]
 22%|██▏       | 2162/10000 [2:33:33<9:19:51,  4.29s/it]
 22%|██▏       | 2163/10000 [2:33:37<9:15:18,  4.25s/it]
 22%|██▏       | 2164/10000 [2:33:42<9:27:11,  4.34s/it]
 22%|██▏       | 2165/10000 [2:33:46<9:21:55,  4.30s/it]
 22%|██▏       | 2166/10000 [2:33:50<9:16:44,  4.26s/it]
 22%|██▏       | 2167/10000 [2:33:55<9:25:16,  4.33s/it]
 22%|██▏       | 2168/10000 [2:33:59<9:17:25,  4.27s/it]
 22%|██▏       | 2169/10000 [2:34:03<9:13:44,  4.24s/it]
 22%|██▏       | 2170/10000 [2:34:08<9:23:30,  4.32s/it]
 22%|██▏       | 2171/10000 [2:34:12<9:31:10,  4.38s/it]
 22%|██▏       | 2172/10000 [2:34:16<9:25:37,  4.34s/it]
 22%|██▏       | 2173/10000 [2:34:21<9:29:57,  4.37s/it]
 22%|██▏       | 2174/10000 [2:34:25<9:21:52,  4.31s/it]
 22%|██▏       | 2175/10000 [2:34:30<9:38:09,  4.43s/it]
 22%|██▏       | 2176/10000 [2:34:34<9:32:02,  4.39s/it]
 22%|██▏       | 2177/10000 [2:34:38<9:23:33,  4.32s/it]
 22%|██▏       | 2178/10000 [2:34:43<9:29:44,  4.37s/it]
 22%|██▏       | 2179/10000 [2:34:47<9:22:13,  4.31s/it]
 22%|██▏       | 2180/10000 [2:34:51<9:17:27,  4.28s/it]
                                                        
{'loss': '0.08304', 'grad_norm': '0.05657', 'learning_rate': '4.001e-06', 'num_tokens': '5.929e+05', 'completions/mean_length': '5.15', 'completions/min_length': '4.5', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.15', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2282', 'rewards/reward_semantic/std': '0.06552', 'rewards/reward_length/mean': '-0.04297', 'rewards/reward_length/std': '0.01263', 'reward': '1.066', 'reward_std': '0.0196', 'frac_reward_zero_std': '0.4', 'kl': '1.867', 'entropy': '0.2317', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.282', 'epoch': '0.3281'}
{'loss': '0.1486', 'grad_norm': '0.007558', 'learning_rate': '3.991e-06', 'num_tokens': '5.991e+05', 'completions/mean_length': '4.675', 'completions/min_length': '4.15', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.675', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2875', 'rewards/reward_semantic/std': '0.02673', 'rewards/reward_length/mean': '-0.03', 'rewards/reward_length/std': '0.005242', 'reward': '1.085', 'reward_std': '0.008207', 'frac_reward_zero_std': '0.65', 'kl': '2.611', 'entropy': '0.1446', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.187', 'epoch': '0.3314'}
{'loss': '0.08427', 'grad_norm': '0.4104', 'learning_rate': '3.981e-06', 'num_tokens': '6.052e+05', 'completions/mean_length': '5.062', 'completions/min_length': '4.6', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.062', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1836', 'rewards/reward_semantic/std': '0.02102', 'rewards/reward_length/mean': '-0.03198', 'rewards/reward_length/std': '0.007345', 'reward': '1.053', 'reward_std': '0.006405', 'frac_reward_zero_std': '0.6', 'kl': '2.375', 'entropy': '0.1376', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.206', 'epoch': '0.3346'}
{'loss': '0.1304', 'grad_norm': '3.887e-05', 'learning_rate': '3.971e-06', 'num_tokens': '6.111e+05', 'completions/mean_length': '4.669', 'completions/min_length': '4.3', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.669', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3293', 'rewards/reward_semantic/std': '0.07271', 'rewards/reward_length/mean': '-0.02915', 'rewards/reward_length/std': '0.007388', 'reward': '1.097', 'reward_std': '0.02163', 'frac_reward_zero_std': '0.65', 'kl': '2.552', 'entropy': '0.1254', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.227', 'epoch': '0.3379'}
{'loss': '0.144', 'grad_norm': '0.04929', 'learning_rate': '3.961e-06', 'num_tokens': '6.171e+05', 'completions/mean_length': '4.231', 'completions/min_length': '3.95', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.231', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3545', 'rewards/reward_semantic/std': '0.03424', 'rewards/reward_length/mean': '-0.02769', 'rewards/reward_length/std': '0.004153', 'reward': '1.105', 'reward_std': '0.01033', 'frac_reward_zero_std': '0.7', 'kl': '3.079', 'entropy': '0.1171', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.268', 'epoch': '0.3412'}
{'loss': '0.07836', 'grad_norm': '0.08275', 'learning_rate': '3.951e-06', 'num_tokens': '6.226e+05', 'completions/mean_length': '4.319', 'completions/min_length': '3.95', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.319', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3827', 'rewards/reward_semantic/std': '0.09277', 'rewards/reward_length/mean': '-0.03306', 'rewards/reward_length/std': '0.008498', 'reward': '1.113', 'reward_std': '0.02744', 'frac_reward_zero_std': '0.65', 'kl': '2.506', 'entropy': '0.1768', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.272', 'epoch': '0.3445'}
{'loss': '0.09549', 'grad_norm': '0.2605', 'learning_rate': '3.941e-06', 'num_tokens': '6.284e+05', 'completions/mean_length': '4.431', 'completions/min_length': '4.25', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.431', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2868', 'rewards/reward_semantic/std': '0.04056', 'rewards/reward_length/mean': '-0.03638', 'rewards/reward_length/std': '0.003325', 'reward': '1.084', 'reward_std': '0.01217', 'frac_reward_zero_std': '0.75', 'kl': '2.272', 'entropy': '0.1447', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.16', 'epoch': '0.3478'}
{'loss': '0.05914', 'grad_norm': '0.03015', 'learning_rate': '3.931e-06', 'num_tokens': '6.341e+05', 'completions/mean_length': '4.306', 'completions/min_length': '3.75', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.306', 'completions/min_terminated_length': '3.75', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3161', 'rewards/reward_semantic/std': '0.06469', 'rewards/reward_length/mean': '-0.03533', 'rewards/reward_length/std': '0.006461', 'reward': '1.093', 'reward_std': '0.01948', 'frac_reward_zero_std': '0.4', 'kl': '2.453', 'entropy': '0.3286', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.226', 'epoch': '0.351'}
{'loss': '0.1025', 'grad_norm': '9.732', 'learning_rate': '3.921e-06', 'num_tokens': '6.4e+05', 'completions/mean_length': '4.469', 'completions/min_length': '4.05', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1892', 'rewards/reward_semantic/std': '0.05419', 'rewards/reward_length/mean': '-0.03215', 'rewards/reward_length/std': '0.01084', 'reward': '1.055', 'reward_std': '0.01634', 'frac_reward_zero_std': '0.5', 'kl': '2.663', 'entropy': '0.2377', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.305', 'epoch': '0.3543'}

 22%|██▏       | 2180/10000 [2:34:51<9:17:27,  4.28s/it]
 22%|██▏       | 2181/10000 [2:34:55<9:25:04,  4.34s/it]
 22%|██▏       | 2182/10000 [2:35:00<9:18:42,  4.29s/it]
 22%|██▏       | 2183/10000 [2:35:04<9:26:12,  4.35s/it]
 22%|██▏       | 2184/10000 [2:35:08<9:19:50,  4.30s/it]
 22%|██▏       | 2185/10000 [2:35:12<9:15:04,  4.26s/it]
 22%|██▏       | 2186/10000 [2:35:17<9:46:29,  4.50s/it]
 22%|██▏       | 2187/10000 [2:35:22<9:38:29,  4.44s/it]
 22%|██▏       | 2188/10000 [2:35:26<9:28:57,  4.37s/it]
 22%|██▏       | 2189/10000 [2:35:31<9:44:57,  4.49s/it]
 22%|██▏       | 2190/10000 [2:35:35<9:41:15,  4.47s/it]
 22%|██▏       | 2191/10000 [2:35:40<9:55:26,  4.57s/it]
 22%|██▏       | 2192/10000 [2:35:44<9:42:58,  4.48s/it]
 22%|██▏       | 2193/10000 [2:35:48<9:29:59,  4.38s/it]
 22%|██▏       | 2194/10000 [2:35:53<9:25:40,  4.35s/it]
 22%|██▏       | 2195/10000 [2:35:57<9:28:02,  4.37s/it]
 22%|██▏       | 2196/10000 [2:36:01<9:21:16,  4.32s/it]
 22%|██▏       | 2197/10000 [2:36:06<9:26:14,  4.35s/it]
 22%|██▏       | 2198/10000 [2:36:10<9:16:28,  4.28s/it]
 22%|██▏       | 2199/10000 [2:36:14<9:13:04,  4.25s/it]
 22%|██▏       | 2200/10000 [2:36:18<9:20:28,  4.31s/it]
                                                        

 22%|██▏       | 2200/10000 [2:36:18<9:20:28,  4.31s/it]
 22%|██▏       | 2201/10000 [2:36:22<9:09:32,  4.23s/it]
 22%|██▏       | 2202/10000 [2:36:27<9:07:12,  4.21s/it]
 22%|██▏       | 2203/10000 [2:36:31<9:16:50,  4.28s/it]
 22%|██▏       | 2204/10000 [2:36:35<9:12:13,  4.25s/it]
 22%|██▏       | 2205/10000 [2:36:39<9:09:07,  4.23s/it]
 22%|██▏       | 2206/10000 [2:36:44<9:21:21,  4.32s/it]
 22%|██▏       | 2207/10000 [2:36:49<9:37:45,  4.45s/it]
 22%|██▏       | 2208/10000 [2:36:53<9:38:26,  4.45s/it]
 22%|██▏       | 2209/10000 [2:36:57<9:26:18,  4.36s/it]
 22%|██▏       | 2210/10000 [2:37:01<9:11:08,  4.24s/it]
 22%|██▏       | 2211/10000 [2:37:06<9:23:02,  4.34s/it]
 22%|██▏       | 2212/10000 [2:37:10<9:16:28,  4.29s/it]
 22%|██▏       | 2213/10000 [2:37:14<9:10:03,  4.24s/it]
 22%|██▏       | 2214/10000 [2:37:19<9:24:43,  4.35s/it]
 22%|██▏       | 2215/10000 [2:37:23<9:19:38,  4.31s/it]
 22%|██▏       | 2216/10000 [2:37:28<9:29:07,  4.39s/it]
 22%|██▏       | 2217/10000 [2:37:32<9:35:00,  4.43s/it]
 22%|██▏       | 2218/10000 [2:37:37<9:33:26,  4.42s/it]
 22%|██▏       | 2219/10000 [2:37:41<9:38:09,  4.46s/it]
 22%|██▏       | 2220/10000 [2:37:45<9:35:10,  4.44s/it]
                                                        

 22%|██▏       | 2220/10000 [2:37:45<9:35:10,  4.44s/it]
 22%|██▏       | 2221/10000 [2:37:50<9:25:47,  4.36s/it]
 22%|██▏       | 2222/10000 [2:37:54<9:22:10,  4.34s/it]
 22%|██▏       | 2223/10000 [2:37:58<9:11:23,  4.25s/it]
 22%|██▏       | 2224/10000 [2:38:02<9:04:15,  4.20s/it]
 22%|██▏       | 2225/10000 [2:38:07<9:26:23,  4.37s/it]
 22%|██▏       | 2226/10000 [2:38:11<9:20:10,  4.32s/it]
 22%|██▏       | 2227/10000 [2:38:15<9:19:14,  4.32s/it]
 22%|██▏       | 2228/10000 [2:38:20<9:27:13,  4.38s/it]
 22%|██▏       | 2229/10000 [2:38:24<9:21:44,  4.34s/it]
 22%|██▏       | 2230/10000 [2:38:28<9:16:34,  4.30s/it]
 22%|██▏       | 2231/10000 [2:38:33<9:24:17,  4.36s/it]
 22%|██▏       | 2232/10000 [2:38:37<9:18:54,  4.32s/it]
 22%|██▏       | 2233/10000 [2:38:42<9:33:59,  4.43s/it]
 22%|██▏       | 2234/10000 [2:38:46<9:25:04,  4.37s/it]
 22%|██▏       | 2235/10000 [2:38:50<9:10:44,  4.26s/it]
 22%|██▏       | 2236/10000 [2:38:54<9:11:51,  4.26s/it]
 22%|██▏       | 2237/10000 [2:38:58<9:08:03,  4.24s/it]
 22%|██▏       | 2238/10000 [2:39:03<9:07:06,  4.23s/it]
 22%|██▏       | 2239/10000 [2:39:07<9:13:59,  4.28s/it]
 22%|██▏       | 2240/10000 [2:39:11<9:12:42,  4.27s/it]
                                                        

 22%|██▏       | 2240/10000 [2:39:11<9:12:42,  4.27s/it]
 22%|██▏       | 2241/10000 [2:39:16<9:12:41,  4.27s/it]
 22%|██▏       | 2242/10000 [2:39:20<9:24:28,  4.37s/it]
 22%|██▏       | 2243/10000 [2:39:29<12:11:40,  5.66s/it]
 22%|██▏       | 2244/10000 [2:39:33<11:31:48,  5.35s/it]
 22%|██▏       | 2245/10000 [2:39:38<11:15:55,  5.23s/it]
 22%|██▏       | 2246/10000 [2:39:43<10:46:10,  5.00s/it]
 22%|██▏       | 2247/10000 [2:39:47<10:21:23,  4.81s/it]
 22%|██▏       | 2248/10000 [2:39:52<10:04:48,  4.68s/it]
 22%|██▏       | 2249/10000 [2:39:56<9:51:27,  4.58s/it] 
 22%|██▎       | 2250/10000 [2:40:00<9:40:38,  4.50s/it]
 23%|██▎       | 2251/10000 [2:40:04<9:27:18,  4.39s/it]
 23%|██▎       | 2252/10000 [2:40:08<9:16:12,  4.31s/it]
 23%|██▎       | 2253/10000 [2:40:13<9:22:57,  4.36s/it]
 23%|██▎       | 2254/10000 [2:40:17<9:19:12,  4.33s/it]
 23%|██▎       | 2255/10000 [2:40:21<9:13:37,  4.29s/it]
 23%|██▎       | 2256/10000 [2:40:26<9:21:11,  4.35s/it]
 23%|██▎       | 2257/10000 [2:40:30<9:15:59,  4.31s/it]
 23%|██▎       | 2258/10000 [2:40:35<9:20:25,  4.34s/it]
 23%|██▎       | 2259/10000 [2:40:39<9:05:49,  4.23s/it]
 23%|██▎       | 2260/10000 [2:40:43<9:01:24,  4.20s/it]
                                                        

 23%|██▎       | 2260/10000 [2:40:43<9:01:24,  4.20s/it]
 23%|██▎       | 2261/10000 [2:40:47<9:12:32,  4.28s/it]
 23%|██▎       | 2262/10000 [2:40:51<9:16:29,  4.32s/it]
 23%|██▎       | 2263/10000 [2:40:56<9:06:11,  4.24s/it]
 23%|██▎       | 2264/10000 [2:41:00<9:16:56,  4.32s/it]
 23%|██▎       | 2265/10000 [2:41:05<9:34:18,  4.45s/it]
 23%|██▎       | 2266/10000 [2:41:09<9:27:37,  4.40s/it]
 23%|██▎       | 2267/10000 [2:41:14<9:34:19,  4.46s/it]
 23%|██▎       | 2268/10000 [2:41:18<9:25:07,  4.39s/it]
 23%|██▎       | 2269/10000 [2:41:22<9:26:50,  4.40s/it]
 23%|██▎       | 2270/10000 [2:41:27<9:31:22,  4.44s/it]
 23%|██▎       | 2271/10000 [2:41:31<9:19:06,  4.34s/it]
 23%|██▎       | 2272/10000 [2:41:35<9:13:07,  4.29s/it]
 23%|██▎       | 2273/10000 [2:41:40<9:23:11,  4.37s/it]
 23%|██▎       | 2274/10000 [2:41:44<9:16:22,  4.32s/it]
 23%|██▎       | 2275/10000 [2:41:48<9:21:16,  4.36s/it]
 23%|██▎       | 2276/10000 [2:41:53<9:18:38,  4.34s/it]
 23%|██▎       | 2277/10000 [2:41:57<9:13:10,  4.30s/it]
 23%|██▎       | 2278/10000 [2:42:01<9:12:00,  4.29s/it]
 23%|██▎       | 2279/10000 [2:42:05<9:08:25,  4.26s/it]
 23%|██▎       | 2280/10000 [2:42:09<8:57:29,  4.18s/it]
                                                        

 23%|██▎       | 2280/10000 [2:42:09<8:57:29,  4.18s/it]
 23%|██▎       | 2281/10000 [2:42:14<9:08:51,  4.27s/it]
 23%|██▎       | 2282/10000 [2:42:18<9:24:54,  4.39s/it]
 23%|██▎       | 2283/10000 [2:42:23<9:17:23,  4.33s/it]
 23%|██▎       | 2284/10000 [2:42:27<9:14:14,  4.31s/it]
 23%|██▎       | 2285/10000 [2:42:31<9:09:32,  4.27s/it]
 23%|██▎       | 2286/10000 [2:42:35<9:05:58,  4.25s/it]
 23%|██▎       | 2287/10000 [2:42:40<9:22:27,  4.38s/it]
 23%|██▎       | 2288/10000 [2:42:44<9:14:54,  4.32s/it]
 23%|██▎       | 2289/10000 [2:42:49<9:20:52,  4.36s/it]
 23%|██▎       | 2290/10000 [2:42:53<9:29:06,  4.43s/it]
 23%|██▎       | 2291/10000 [2:42:57<9:19:13,  4.35s/it]
 23%|██▎       | 2292/10000 [2:43:02<9:23:34,  4.39s/it]
 23%|██▎       | 2293/10000 [2:43:06<9:16:02,  4.33s/it]
 23%|██▎       | 2294/10000 [2:43:10<9:10:18,  4.28s/it]
 23%|██▎       | 2295/10000 [2:43:15<9:17:40,  4.34s/it]
 23%|██▎       | 2296/10000 [2:43:19<9:11:44,  4.30s/it]
 23%|██▎       | 2297/10000 [2:43:23<9:07:36,  4.27s/it]
 23%|██▎       | 2298/10000 [2:43:28<9:19:32,  4.36s/it]
 23%|██▎       | 2299/10000 [2:43:32<9:13:02,  4.31s/it]
 23%|██▎       | 2300/10000 [2:43:36<9:08:44,  4.28s/it]
                                                        

 23%|██▎       | 2300/10000 [2:43:36<9:08:44,  4.28s/it]
 23%|██▎       | 2301/10000 [2:43:41<9:17:25,  4.34s/it]
 23%|██▎       | 2302/10000 [2:43:45<9:15:51,  4.33s/it]
 23%|██▎       | 2303/10000 [2:43:49<9:20:52,  4.37s/it]
 23%|██▎       | 2304/10000 [2:43:54<9:14:01,  4.32s/it]
 23%|██▎       | 2305/10000 [2:43:58<9:12:58,  4.31s/it]
 23%|██▎       | 2306/10000 [2:44:02<9:19:46,  4.37s/it]
 23%|██▎       | 2307/10000 [2:44:06<9:12:34,  4.31s/it]
 23%|██▎       | 2308/10000 [2:44:11<9:08:23,  4.28s/it]
 23%|██▎       | 2309/10000 [2:44:15<9:23:32,  4.40s/it]
 23%|██▎       | 2310/10000 [2:44:20<9:16:38,  4.34s/it]
 23%|██▎       | 2311/10000 [2:44:24<9:10:59,  4.30s/it]
 23%|██▎       | 2312/10000 [2:44:28<9:23:05,  4.39s/it]
 23%|██▎       | 2313/10000 [2:44:33<9:12:45,  4.31s/it]
 23%|██▎       | 2314/10000 [2:44:37<9:19:25,  4.37s/it]
 23%|██▎       | 2315/10000 [2:44:41<9:13:02,  4.32s/it]
 23%|██▎       | 2316/10000 [2:44:45<9:09:14,  4.29s/it]
 23%|██▎       | 2317/10000 [2:44:50<9:16:51,  4.35s/it]
 23%|██▎       | 2318/10000 [2:44:54<9:11:02,  4.30s/it]
 23%|██▎       | 2319/10000 [2:44:58<9:06:57,  4.27s/it]
 23%|██▎       | 2320/10000 [2:45:03<9:07:01,  4.27s/it]
                                                        

 23%|██▎       | 2320/10000 [2:45:03<9:07:01,  4.27s/it]
 23%|██▎       | 2321/10000 [2:45:07<9:04:11,  4.25s/it]
 23%|██▎       | 2322/10000 [2:45:11<9:02:12,  4.24s/it]
 23%|██▎       | 2323/10000 [2:45:15<9:03:20,  4.25s/it]
 23%|██▎       | 2324/10000 [2:45:19<9:01:44,  4.23s/it]
 23%|██▎       | 2325/10000 [2:45:24<9:13:53,  4.33s/it]
 23%|██▎       | 2326/10000 [2:45:29<9:20:11,  4.38s/it]
 23%|██▎       | 2327/10000 [2:45:33<9:13:06,  4.33s/it]
 23%|██▎       | 2328/10000 [2:45:37<9:08:07,  4.29s/it]
 23%|██▎       | 2329/10000 [2:45:42<9:19:17,  4.37s/it]
 23%|██▎       | 2330/10000 [2:45:46<9:13:03,  4.33s/it]
 23%|██▎       | 2331/10000 [2:45:50<9:10:59,  4.31s/it]
 23%|██▎       | 2332/10000 [2:45:54<9:06:18,  4.27s/it]
 23%|██▎       | 2333/10000 [2:45:58<9:02:58,  4.25s/it]
 23%|██▎       | 2334/10000 [2:46:03<9:12:27,  4.32s/it]
 23%|██▎       | 2335/10000 [2:46:07<9:01:43,  4.24s/it]
 23%|██▎       | 2336/10000 [2:46:11<9:00:05,  4.23s/it]
 23%|██▎       | 2337/10000 [2:46:15<9:01:33,  4.24s/it]
 23%|██▎       | 2338/10000 [2:46:20<9:12:43,  4.33s/it]
 23%|██▎       | 2339/10000 [2:46:24<9:07:24,  4.29s/it]
 23%|██▎       | 2340/10000 [2:46:29<9:18:35,  4.38s/it]
                                                        

 23%|██▎       | 2340/10000 [2:46:29<9:18:35,  4.38s/it]
 23%|██▎       | 2341/10000 [2:46:33<9:11:43,  4.32s/it]
 23%|██▎       | 2342/10000 [2:46:37<9:06:44,  4.28s/it]
 23%|██▎       | 2343/10000 [2:46:42<9:19:44,  4.39s/it]
 23%|██▎       | 2344/10000 [2:46:46<9:12:38,  4.33s/it]
 23%|██▎       | 2345/10000 [2:46:50<9:12:50,  4.33s/it]
 23%|██▎       | 2346/10000 [2:46:54<9:06:17,  4.28s/it]
 23%|██▎       | 2347/10000 [2:46:59<9:01:31,  4.25s/it]
 23%|██▎       | 2348/10000 [2:47:03<9:10:35,  4.32s/it]
 23%|██▎       | 2349/10000 [2:47:08<9:22:46,  4.41s/it]
 24%|██▎       | 2350/10000 [2:47:12<9:15:10,  4.35s/it]
 24%|██▎       | 2351/10000 [2:47:16<9:20:34,  4.40s/it]
 24%|██▎       | 2352/10000 [2:47:21<9:18:33,  4.38s/it]
 24%|██▎       | 2353/10000 [2:47:25<9:12:24,  4.33s/it]
 24%|██▎       | 2354/10000 [2:47:30<9:26:23,  4.44s/it]
 24%|██▎       | 2355/10000 [2:47:34<9:16:01,  4.36s/it]
 24%|██▎       | 2356/10000 [2:47:38<9:20:10,  4.40s/it]
 24%|██▎       | 2357/10000 [2:47:43<9:12:54,  4.34s/it]
 24%|██▎       | 2358/10000 [2:47:47<9:07:16,  4.30s/it]
 24%|██▎       | 2359/10000 [2:47:51<9:14:36,  4.36s/it]
 24%|██▎       | 2360/10000 [2:47:55<9:08:14,  4.31s/it]
                                                        
{'loss': '0.1483', 'grad_norm': '0.06326', 'learning_rate': '3.911e-06', 'num_tokens': '6.461e+05', 'completions/mean_length': '4.669', 'completions/min_length': '4.15', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.669', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3036', 'rewards/reward_semantic/std': '0.0768', 'rewards/reward_length/mean': '-0.02412', 'rewards/reward_length/std': '0.005769', 'reward': '1.09', 'reward_std': '0.02297', 'frac_reward_zero_std': '0.6', 'kl': '2.389', 'entropy': '0.2346', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.302', 'epoch': '0.3576'}
{'loss': '0.05962', 'grad_norm': '9.089', 'learning_rate': '3.901e-06', 'num_tokens': '6.519e+05', 'completions/mean_length': '4.694', 'completions/min_length': '3.65', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.694', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2612', 'rewards/reward_semantic/std': '0.1004', 'rewards/reward_length/mean': '-0.03782', 'rewards/reward_length/std': '0.01144', 'reward': '1.076', 'reward_std': '0.03019', 'frac_reward_zero_std': '0.45', 'kl': '2.234', 'entropy': '0.2023', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.358', 'epoch': '0.3609'}
{'loss': '0.1325', 'grad_norm': '19.8', 'learning_rate': '3.891e-06', 'num_tokens': '6.58e+05', 'completions/mean_length': '4.469', 'completions/min_length': '3.85', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '3.85', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3336', 'rewards/reward_semantic/std': '0.1172', 'rewards/reward_length/mean': '-0.03862', 'rewards/reward_length/std': '0.008906', 'reward': '1.098', 'reward_std': '0.03529', 'frac_reward_zero_std': '0.5', 'kl': '2.147', 'entropy': '0.2475', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.329', 'epoch': '0.3642'}
{'loss': '0.081', 'grad_norm': '0.007902', 'learning_rate': '3.881e-06', 'num_tokens': '6.639e+05', 'completions/mean_length': '4.388', 'completions/min_length': '4.05', 'completions/max_length': '4.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.388', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4348', 'rewards/reward_semantic/std': '0.06267', 'rewards/reward_length/mean': '-0.03401', 'rewards/reward_length/std': '0.004638', 'reward': '1.129', 'reward_std': '0.01874', 'frac_reward_zero_std': '0.7', 'kl': '1.992', 'entropy': '0.09938', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.271', 'epoch': '0.3675'}
{'loss': '0.1362', 'grad_norm': '13.6', 'learning_rate': '3.871e-06', 'num_tokens': '6.699e+05', 'completions/mean_length': '5.075', 'completions/min_length': '4.25', 'completions/max_length': '8.4', 'completions/clipped_ratio': '0.00625', 'completions/mean_terminated_length': '4.697', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '5.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.366', 'rewards/reward_semantic/std': '0.09823', 'rewards/reward_length/mean': '-0.05349', 'rewards/reward_length/std': '0.02343', 'reward': '1.107', 'reward_std': '0.0299', 'frac_reward_zero_std': '0.5', 'kl': '1.917', 'entropy': '0.2259', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.549', 'epoch': '0.3707'}
{'loss': '0.1078', 'grad_norm': '0.1943', 'learning_rate': '3.861e-06', 'num_tokens': '6.757e+05', 'completions/mean_length': '4.494', 'completions/min_length': '4', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.494', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2344', 'rewards/reward_semantic/std': '0.03166', 'rewards/reward_length/mean': '-0.0373', 'rewards/reward_length/std': '0.01196', 'reward': '1.068', 'reward_std': '0.009906', 'frac_reward_zero_std': '0.5', 'kl': '2.282', 'entropy': '0.184', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.315', 'epoch': '0.374'}
{'loss': '0.1381', 'grad_norm': '21.58', 'learning_rate': '3.851e-06', 'num_tokens': '6.816e+05', 'completions/mean_length': '4.812', 'completions/min_length': '4.45', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.812', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1875', 'rewards/reward_semantic/std': '0.03536', 'rewards/reward_length/mean': '-0.02378', 'rewards/reward_length/std': '0.003466', 'reward': '1.055', 'reward_std': '0.01078', 'frac_reward_zero_std': '0.75', 'kl': '2.779', 'entropy': '0.102', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.317', 'epoch': '0.3773'}
{'loss': '0.1417', 'grad_norm': '0.06855', 'learning_rate': '3.841e-06', 'num_tokens': '6.875e+05', 'completions/mean_length': '4.475', 'completions/min_length': '4.2', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.475', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4098', 'rewards/reward_semantic/std': '0.08722', 'rewards/reward_length/mean': '-0.02837', 'rewards/reward_length/std': '0.006398', 'reward': '1.122', 'reward_std': '0.02616', 'frac_reward_zero_std': '0.6', 'kl': '2.662', 'entropy': '0.1086', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.307', 'epoch': '0.3806'}
{'loss': '0.124', 'grad_norm': '35.62', 'learning_rate': '3.831e-06', 'num_tokens': '6.934e+05', 'completions/mean_length': '4.412', 'completions/min_length': '4.3', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.412', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.335', 'rewards/reward_semantic/std': '0.07739', 'rewards/reward_length/mean': '-0.02791', 'rewards/reward_length/std': '0.005625', 'reward': '1.099', 'reward_std': '0.02316', 'frac_reward_zero_std': '0.7', 'kl': '3.274', 'entropy': '0.1248', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.285', 'epoch': '0.3839'}

 24%|██▎       | 2360/10000 [2:47:55<9:08:14,  4.31s/it]
 24%|██▎       | 2361/10000 [2:48:00<9:01:19,  4.25s/it]
 24%|██▎       | 2362/10000 [2:48:04<9:10:15,  4.32s/it]
 24%|██▎       | 2363/10000 [2:48:08<8:57:20,  4.22s/it]
 24%|██▎       | 2364/10000 [2:48:12<8:55:55,  4.21s/it]
 24%|██▎       | 2365/10000 [2:48:17<9:07:15,  4.30s/it]
 24%|██▎       | 2366/10000 [2:48:21<9:04:18,  4.28s/it]
 24%|██▎       | 2367/10000 [2:48:25<9:12:26,  4.34s/it]
 24%|██▎       | 2368/10000 [2:48:29<8:59:10,  4.24s/it]
 24%|██▎       | 2369/10000 [2:48:34<8:57:55,  4.23s/it]
 24%|██▎       | 2370/10000 [2:48:38<9:14:18,  4.36s/it]
 24%|██▎       | 2371/10000 [2:48:42<9:07:49,  4.31s/it]
 24%|██▎       | 2372/10000 [2:48:47<9:04:37,  4.28s/it]
 24%|██▎       | 2373/10000 [2:48:51<9:11:19,  4.34s/it]
 24%|██▎       | 2374/10000 [2:48:55<9:08:34,  4.32s/it]
 24%|██▍       | 2375/10000 [2:49:00<9:15:15,  4.37s/it]
 24%|██▍       | 2376/10000 [2:49:04<9:00:34,  4.25s/it]
 24%|██▍       | 2377/10000 [2:49:08<9:08:59,  4.32s/it]
 24%|██▍       | 2378/10000 [2:49:13<9:15:29,  4.37s/it]
 24%|██▍       | 2379/10000 [2:49:17<9:09:05,  4.32s/it]
 24%|██▍       | 2380/10000 [2:49:21<9:05:06,  4.29s/it]
                                                        

 24%|██▍       | 2380/10000 [2:49:21<9:05:06,  4.29s/it]
 24%|██▍       | 2381/10000 [2:49:26<9:12:47,  4.35s/it]
 24%|██▍       | 2382/10000 [2:49:30<9:07:05,  4.31s/it]
 24%|██▍       | 2383/10000 [2:49:34<9:05:57,  4.30s/it]
 24%|██▍       | 2384/10000 [2:49:39<9:07:15,  4.31s/it]
 24%|██▍       | 2385/10000 [2:49:43<9:03:08,  4.28s/it]
 24%|██▍       | 2386/10000 [2:49:47<9:11:09,  4.34s/it]
 24%|██▍       | 2387/10000 [2:49:52<9:05:52,  4.30s/it]
 24%|██▍       | 2388/10000 [2:49:56<9:12:43,  4.36s/it]
 24%|██▍       | 2389/10000 [2:50:00<9:07:01,  4.31s/it]
 24%|██▍       | 2390/10000 [2:50:04<9:03:01,  4.28s/it]
 24%|██▍       | 2391/10000 [2:50:09<9:11:17,  4.35s/it]
 24%|██▍       | 2392/10000 [2:50:13<9:05:29,  4.30s/it]
 24%|██▍       | 2393/10000 [2:50:17<8:58:50,  4.25s/it]
 24%|██▍       | 2394/10000 [2:50:22<9:07:48,  4.32s/it]
 24%|██▍       | 2395/10000 [2:50:26<9:00:00,  4.26s/it]
 24%|██▍       | 2396/10000 [2:50:30<9:08:43,  4.33s/it]
 24%|██▍       | 2397/10000 [2:50:35<9:03:21,  4.29s/it]
 24%|██▍       | 2398/10000 [2:50:39<8:57:04,  4.24s/it]
 24%|██▍       | 2399/10000 [2:50:43<9:06:20,  4.31s/it]
 24%|██▍       | 2400/10000 [2:50:47<9:02:24,  4.28s/it]
                                                        

 24%|██▍       | 2400/10000 [2:50:47<9:02:24,  4.28s/it]
 24%|██▍       | 2401/10000 [2:50:52<9:02:46,  4.29s/it]
 24%|██▍       | 2402/10000 [2:50:56<9:10:46,  4.35s/it]
 24%|██▍       | 2403/10000 [2:51:00<9:05:08,  4.31s/it]
 24%|██▍       | 2404/10000 [2:51:05<9:11:57,  4.36s/it]
 24%|██▍       | 2405/10000 [2:51:09<9:05:59,  4.31s/it]
 24%|██▍       | 2406/10000 [2:51:13<9:01:49,  4.28s/it]
 24%|██▍       | 2407/10000 [2:51:18<9:09:59,  4.35s/it]
 24%|██▍       | 2408/10000 [2:51:22<9:09:13,  4.34s/it]
 24%|██▍       | 2409/10000 [2:51:27<9:15:06,  4.39s/it]
 24%|██▍       | 2410/10000 [2:51:31<9:07:49,  4.33s/it]
 24%|██▍       | 2411/10000 [2:51:35<9:07:12,  4.33s/it]
 24%|██▍       | 2412/10000 [2:51:40<9:13:34,  4.38s/it]
 24%|██▍       | 2413/10000 [2:51:44<9:06:38,  4.32s/it]
 24%|██▍       | 2414/10000 [2:51:48<9:06:51,  4.33s/it]
 24%|██▍       | 2415/10000 [2:51:53<9:12:54,  4.37s/it]
 24%|██▍       | 2416/10000 [2:51:57<8:57:39,  4.25s/it]
 24%|██▍       | 2417/10000 [2:52:01<9:06:30,  4.32s/it]
 24%|██▍       | 2418/10000 [2:52:05<8:59:49,  4.27s/it]
 24%|██▍       | 2419/10000 [2:52:09<8:57:18,  4.25s/it]
 24%|██▍       | 2420/10000 [2:52:14<9:04:57,  4.31s/it]
                                                        

 24%|██▍       | 2420/10000 [2:52:14<9:04:57,  4.31s/it]
 24%|██▍       | 2421/10000 [2:52:18<9:00:48,  4.28s/it]
 24%|██▍       | 2422/10000 [2:52:23<9:09:14,  4.35s/it]
 24%|██▍       | 2423/10000 [2:52:27<9:00:54,  4.28s/it]
 24%|██▍       | 2424/10000 [2:52:31<8:49:37,  4.19s/it]
 24%|██▍       | 2425/10000 [2:52:35<9:02:30,  4.30s/it]
 24%|██▍       | 2426/10000 [2:52:39<8:57:26,  4.26s/it]
 24%|██▍       | 2427/10000 [2:52:44<8:55:19,  4.24s/it]
 24%|██▍       | 2428/10000 [2:52:48<9:07:47,  4.34s/it]
 24%|██▍       | 2429/10000 [2:52:52<9:00:01,  4.28s/it]
 24%|██▍       | 2430/10000 [2:52:57<9:06:48,  4.33s/it]
 24%|██▍       | 2431/10000 [2:53:01<9:01:32,  4.29s/it]
 24%|██▍       | 2432/10000 [2:53:05<8:58:07,  4.27s/it]
 24%|██▍       | 2433/10000 [2:53:10<9:06:47,  4.34s/it]
 24%|██▍       | 2434/10000 [2:53:14<9:01:00,  4.29s/it]
 24%|██▍       | 2435/10000 [2:53:18<8:59:05,  4.28s/it]
 24%|██▍       | 2436/10000 [2:53:23<9:31:16,  4.53s/it]
 24%|██▍       | 2437/10000 [2:53:27<9:19:03,  4.44s/it]
 24%|██▍       | 2438/10000 [2:53:32<9:20:39,  4.45s/it]
 24%|██▍       | 2439/10000 [2:53:36<9:10:20,  4.37s/it]
 24%|██▍       | 2440/10000 [2:53:40<9:03:27,  4.31s/it]
                                                        

 24%|██▍       | 2440/10000 [2:53:40<9:03:27,  4.31s/it]
 24%|██▍       | 2441/10000 [2:53:45<9:13:48,  4.40s/it]
 24%|██▍       | 2442/10000 [2:53:49<9:04:12,  4.32s/it]
 24%|██▍       | 2443/10000 [2:53:53<9:02:18,  4.31s/it]
 24%|██▍       | 2444/10000 [2:53:58<9:08:14,  4.35s/it]
 24%|██▍       | 2445/10000 [2:54:02<9:01:05,  4.30s/it]
 24%|██▍       | 2446/10000 [2:54:06<8:56:58,  4.27s/it]
 24%|██▍       | 2447/10000 [2:54:11<9:04:12,  4.32s/it]
 24%|██▍       | 2448/10000 [2:54:15<8:59:02,  4.28s/it]
 24%|██▍       | 2449/10000 [2:54:19<9:05:49,  4.34s/it]
 24%|██▍       | 2450/10000 [2:54:24<9:11:05,  4.38s/it]
 25%|██▍       | 2451/10000 [2:54:28<9:03:21,  4.32s/it]
 25%|██▍       | 2452/10000 [2:54:32<9:08:52,  4.36s/it]
 25%|██▍       | 2453/10000 [2:54:37<9:25:54,  4.50s/it]
 25%|██▍       | 2454/10000 [2:54:42<9:24:34,  4.49s/it]
 25%|██▍       | 2455/10000 [2:54:46<9:12:29,  4.39s/it]
 25%|██▍       | 2456/10000 [2:54:50<9:04:20,  4.33s/it]
 25%|██▍       | 2457/10000 [2:54:54<9:09:09,  4.37s/it]
 25%|██▍       | 2458/10000 [2:54:59<9:01:38,  4.31s/it]
 25%|██▍       | 2459/10000 [2:55:03<8:55:19,  4.26s/it]
 25%|██▍       | 2460/10000 [2:55:07<9:04:57,  4.34s/it]
                                                        

 25%|██▍       | 2460/10000 [2:55:07<9:04:57,  4.34s/it]
 25%|██▍       | 2461/10000 [2:55:11<8:59:20,  4.29s/it]
 25%|██▍       | 2462/10000 [2:55:16<8:56:13,  4.27s/it]
 25%|██▍       | 2463/10000 [2:55:20<8:52:15,  4.24s/it]
 25%|██▍       | 2464/10000 [2:55:24<8:54:47,  4.26s/it]
 25%|██▍       | 2465/10000 [2:55:29<9:02:45,  4.32s/it]
 25%|██▍       | 2466/10000 [2:55:33<8:53:39,  4.25s/it]
 25%|██▍       | 2467/10000 [2:55:37<8:46:23,  4.19s/it]
 25%|██▍       | 2468/10000 [2:55:41<8:52:52,  4.24s/it]
 25%|██▍       | 2469/10000 [2:55:45<8:46:25,  4.19s/it]
 25%|██▍       | 2470/10000 [2:55:50<8:51:33,  4.24s/it]
 25%|██▍       | 2471/10000 [2:55:54<8:45:15,  4.19s/it]
 25%|██▍       | 2472/10000 [2:55:58<8:40:21,  4.15s/it]
 25%|██▍       | 2473/10000 [2:56:02<8:48:05,  4.21s/it]
 25%|██▍       | 2474/10000 [2:56:06<8:42:37,  4.17s/it]
 25%|██▍       | 2475/10000 [2:56:10<8:49:21,  4.22s/it]
 25%|██▍       | 2476/10000 [2:56:15<8:43:20,  4.17s/it]
 25%|██▍       | 2477/10000 [2:56:19<8:36:37,  4.12s/it]
 25%|██▍       | 2478/10000 [2:56:23<8:45:21,  4.19s/it]
 25%|██▍       | 2479/10000 [2:56:27<8:40:10,  4.15s/it]
 25%|██▍       | 2480/10000 [2:56:31<8:44:14,  4.18s/it]
                                                        

 25%|██▍       | 2480/10000 [2:56:31<8:44:14,  4.18s/it]
 25%|██▍       | 2481/10000 [2:56:36<8:50:21,  4.23s/it]
 25%|██▍       | 2482/10000 [2:56:40<8:41:34,  4.16s/it]
 25%|██▍       | 2483/10000 [2:56:44<8:53:36,  4.26s/it]
 25%|██▍       | 2484/10000 [2:56:48<8:45:59,  4.20s/it]
 25%|██▍       | 2485/10000 [2:56:52<8:45:29,  4.20s/it]
 25%|██▍       | 2486/10000 [2:56:57<8:51:27,  4.24s/it]
 25%|██▍       | 2487/10000 [2:57:01<8:48:02,  4.22s/it]
 25%|██▍       | 2488/10000 [2:57:05<8:44:19,  4.19s/it]
 25%|██▍       | 2489/10000 [2:57:09<8:54:23,  4.27s/it]
 25%|██▍       | 2490/10000 [2:57:13<8:45:57,  4.20s/it]
 25%|██▍       | 2491/10000 [2:57:17<8:40:32,  4.16s/it]
 25%|██▍       | 2492/10000 [2:57:22<8:47:20,  4.21s/it]
 25%|██▍       | 2493/10000 [2:57:26<8:40:39,  4.16s/it]
 25%|██▍       | 2494/10000 [2:57:30<8:47:45,  4.22s/it]
 25%|██▍       | 2495/10000 [2:57:35<9:04:53,  4.36s/it]
 25%|██▍       | 2496/10000 [2:57:39<8:53:12,  4.26s/it]
 25%|██▍       | 2497/10000 [2:57:43<8:53:38,  4.27s/it]
 25%|██▍       | 2498/10000 [2:57:47<8:46:35,  4.21s/it]
 25%|██▍       | 2499/10000 [2:57:51<8:40:57,  4.17s/it]
 25%|██▌       | 2500/10000 [2:57:56<8:47:53,  4.22s/it]
                                                        

 25%|██▌       | 2500/10000 [2:57:56<8:47:53,  4.22s/it]
 25%|██▌       | 2501/10000 [2:58:01<9:23:07,  4.51s/it]
 25%|██▌       | 2502/10000 [2:58:05<9:19:02,  4.47s/it]
 25%|██▌       | 2503/10000 [2:58:09<9:07:00,  4.38s/it]
 25%|██▌       | 2504/10000 [2:58:13<8:55:15,  4.28s/it]
 25%|██▌       | 2505/10000 [2:58:18<8:58:26,  4.31s/it]
 25%|██▌       | 2506/10000 [2:58:22<8:49:36,  4.24s/it]
 25%|██▌       | 2507/10000 [2:58:26<8:43:38,  4.19s/it]
 25%|██▌       | 2508/10000 [2:58:30<8:47:11,  4.22s/it]
 25%|██▌       | 2509/10000 [2:58:34<8:41:24,  4.18s/it]
 25%|██▌       | 2510/10000 [2:58:38<8:36:32,  4.14s/it]
 25%|██▌       | 2511/10000 [2:58:43<8:43:10,  4.19s/it]
 25%|██▌       | 2512/10000 [2:58:47<8:38:30,  4.15s/it]
 25%|██▌       | 2513/10000 [2:58:51<8:45:48,  4.21s/it]
 25%|██▌       | 2514/10000 [2:58:55<8:39:19,  4.16s/it]
 25%|██▌       | 2515/10000 [2:58:59<8:32:51,  4.11s/it]
 25%|██▌       | 2516/10000 [2:59:04<9:00:36,  4.33s/it]
 25%|██▌       | 2517/10000 [2:59:08<9:03:29,  4.36s/it]
 25%|██▌       | 2518/10000 [2:59:13<9:04:15,  4.36s/it]
 25%|██▌       | 2519/10000 [2:59:17<8:53:48,  4.28s/it]
 25%|██▌       | 2520/10000 [2:59:21<8:49:40,  4.25s/it]
                                                        

 25%|██▌       | 2520/10000 [2:59:21<8:49:40,  4.25s/it]
 25%|██▌       | 2521/10000 [2:59:25<8:53:59,  4.28s/it]
 25%|██▌       | 2522/10000 [2:59:30<8:57:25,  4.31s/it]
 25%|██▌       | 2523/10000 [2:59:34<8:49:52,  4.25s/it]
 25%|██▌       | 2524/10000 [2:59:38<8:49:11,  4.25s/it]
 25%|██▌       | 2525/10000 [2:59:42<8:46:50,  4.23s/it]
 25%|██▌       | 2526/10000 [2:59:46<8:42:25,  4.19s/it]
 25%|██▌       | 2527/10000 [2:59:51<8:54:48,  4.29s/it]
 25%|██▌       | 2528/10000 [2:59:55<8:42:19,  4.19s/it]
 25%|██▌       | 2529/10000 [2:59:59<8:50:41,  4.26s/it]
 25%|██▌       | 2530/10000 [3:00:04<8:52:00,  4.27s/it]
 25%|██▌       | 2531/10000 [3:00:08<8:40:47,  4.18s/it]
 25%|██▌       | 2532/10000 [3:00:12<8:48:56,  4.25s/it]
 25%|██▌       | 2533/10000 [3:00:16<8:46:34,  4.23s/it]
 25%|██▌       | 2534/10000 [3:00:21<8:59:05,  4.33s/it]
 25%|██▌       | 2535/10000 [3:00:25<8:59:54,  4.34s/it]
 25%|██▌       | 2536/10000 [3:00:29<8:45:26,  4.22s/it]
 25%|██▌       | 2537/10000 [3:00:34<9:08:06,  4.41s/it]
 25%|██▌       | 2538/10000 [3:00:38<9:09:58,  4.42s/it]
 25%|██▌       | 2539/10000 [3:00:43<9:12:00,  4.44s/it]
 25%|██▌       | 2540/10000 [3:00:47<9:04:09,  4.38s/it]
                                                        
{'loss': '0.1043', 'grad_norm': '0.05648', 'learning_rate': '3.821e-06', 'num_tokens': '6.992e+05', 'completions/mean_length': '4.756', 'completions/min_length': '4.5', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.756', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.05942', 'rewards/reward_semantic/std': '0.02301', 'rewards/reward_length/mean': '-0.02983', 'rewards/reward_length/std': '0.007803', 'reward': '1.016', 'reward_std': '0.007185', 'frac_reward_zero_std': '0.65', 'kl': '2.363', 'entropy': '0.1625', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.316', 'epoch': '0.3871'}
{'loss': '0.114', 'grad_norm': '5.801', 'learning_rate': '3.811e-06', 'num_tokens': '7.05e+05', 'completions/mean_length': '4.438', 'completions/min_length': '4', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.438', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3108', 'rewards/reward_semantic/std': '0.004481', 'rewards/reward_length/mean': '-0.02412', 'rewards/reward_length/std': '0.004905', 'reward': '1.092', 'reward_std': '0.00143', 'frac_reward_zero_std': '0.7', 'kl': '3.305', 'entropy': '0.09237', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.275', 'epoch': '0.3904'}
{'loss': '0.1353', 'grad_norm': '0.002456', 'learning_rate': '3.801e-06', 'num_tokens': '7.109e+05', 'completions/mean_length': '4.619', 'completions/min_length': '4.5', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.619', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3063', 'rewards/reward_semantic/std': '0.04082', 'rewards/reward_length/mean': '-0.02515', 'rewards/reward_length/std': '0.001519', 'reward': '1.091', 'reward_std': '0.01223', 'frac_reward_zero_std': '0.85', 'kl': '2.924', 'entropy': '0.04164', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.283', 'epoch': '0.3937'}
{'loss': '0.1441', 'grad_norm': '0.0061', 'learning_rate': '3.791e-06', 'num_tokens': '7.168e+05', 'completions/mean_length': '4.544', 'completions/min_length': '4.5', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.544', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2875', 'rewards/reward_semantic/std': '0.03536', 'rewards/reward_length/mean': '-0.02251', 'rewards/reward_length/std': '0.001057', 'reward': '1.085', 'reward_std': '0.0106', 'frac_reward_zero_std': '0.8', 'kl': '3.01', 'entropy': '0.05115', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.306', 'epoch': '0.397'}
{'loss': '0.1032', 'grad_norm': '0.000867', 'learning_rate': '3.781e-06', 'num_tokens': '7.226e+05', 'completions/mean_length': '4.862', 'completions/min_length': '4.5', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.862', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.25', 'rewards/reward_semantic/std': '0', 'rewards/reward_length/mean': '-0.02761', 'rewards/reward_length/std': '0.00345', 'reward': '1.074', 'reward_std': '0.0001725', 'frac_reward_zero_std': '0.8', 'kl': '2.889', 'entropy': '0.09566', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.301', 'epoch': '0.4003'}
{'loss': '0.138', 'grad_norm': '0.0003204', 'learning_rate': '3.771e-06', 'num_tokens': '7.285e+05', 'completions/mean_length': '4.987', 'completions/min_length': '4.55', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.987', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3438', 'rewards/reward_semantic/std': '0.02539', 'rewards/reward_length/mean': '-0.02656', 'rewards/reward_length/std': '0.005759', 'reward': '1.102', 'reward_std': '0.007857', 'frac_reward_zero_std': '0.7', 'kl': '3.108', 'entropy': '0.1048', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.329', 'epoch': '0.4035'}
{'loss': '0.1579', 'grad_norm': '12.83', 'learning_rate': '3.761e-06', 'num_tokens': '7.343e+05', 'completions/mean_length': '4.569', 'completions/min_length': '4.4', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.569', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1531', 'rewards/reward_semantic/std': '0.02652', 'rewards/reward_length/mean': '-0.0186', 'rewards/reward_length/std': '0.001243', 'reward': '1.045', 'reward_std': '0.008003', 'frac_reward_zero_std': '0.85', 'kl': '3.267', 'entropy': '0.03913', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.176', 'epoch': '0.4068'}
{'loss': '0.1812', 'grad_norm': '0.0002541', 'learning_rate': '3.751e-06', 'num_tokens': '7.401e+05', 'completions/mean_length': '4.544', 'completions/min_length': '4.1', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.544', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3563', 'rewards/reward_semantic/std': '0.08315', 'rewards/reward_length/mean': '-0.0261', 'rewards/reward_length/std': '0.004345', 'reward': '1.106', 'reward_std': '0.02494', 'frac_reward_zero_std': '0.7', 'kl': '2.858', 'entropy': '0.1132', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.207', 'epoch': '0.4101'}
{'loss': '0.1174', 'grad_norm': '20.05', 'learning_rate': '3.741e-06', 'num_tokens': '7.461e+05', 'completions/mean_length': '4.731', 'completions/min_length': '4.15', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.731', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4852', 'rewards/reward_semantic/std': '0.124', 'rewards/reward_length/mean': '-0.03533', 'rewards/reward_length/std': '0.01076', 'reward': '1.144', 'reward_std': '0.03678', 'frac_reward_zero_std': '0.55', 'kl': '2.782', 'entropy': '0.1758', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.2', 'epoch': '0.4134'}

 25%|██▌       | 2540/10000 [3:00:47<9:04:09,  4.38s/it]
 25%|██▌       | 2541/10000 [3:00:51<8:58:42,  4.33s/it]
 25%|██▌       | 2542/10000 [3:00:56<9:03:33,  4.37s/it]
 25%|██▌       | 2543/10000 [3:01:00<8:56:30,  4.32s/it]
 25%|██▌       | 2544/10000 [3:01:04<8:43:27,  4.21s/it]
 25%|██▌       | 2545/10000 [3:01:09<9:06:26,  4.40s/it]
 25%|██▌       | 2546/10000 [3:01:13<8:58:01,  4.33s/it]
 25%|██▌       | 2547/10000 [3:01:18<9:16:21,  4.48s/it]
 25%|██▌       | 2548/10000 [3:01:22<9:08:07,  4.41s/it]
 25%|██▌       | 2549/10000 [3:01:26<9:00:53,  4.36s/it]
 26%|██▌       | 2550/10000 [3:01:31<9:05:16,  4.39s/it]
 26%|██▌       | 2551/10000 [3:01:35<8:57:19,  4.33s/it]
 26%|██▌       | 2552/10000 [3:01:39<8:51:47,  4.28s/it]
 26%|██▌       | 2553/10000 [3:01:44<8:58:37,  4.34s/it]
 26%|██▌       | 2554/10000 [3:01:48<8:52:25,  4.29s/it]
 26%|██▌       | 2555/10000 [3:01:52<8:50:17,  4.27s/it]
 26%|██▌       | 2556/10000 [3:01:56<8:46:53,  4.25s/it]
 26%|██▌       | 2557/10000 [3:02:01<9:07:22,  4.41s/it]
 26%|██▌       | 2558/10000 [3:02:05<9:09:15,  4.43s/it]
 26%|██▌       | 2559/10000 [3:02:10<9:04:03,  4.39s/it]
 26%|██▌       | 2560/10000 [3:02:14<9:07:02,  4.41s/it]
                                                        

 26%|██▌       | 2560/10000 [3:02:14<9:07:02,  4.41s/it]
 26%|██▌       | 2561/10000 [3:02:18<8:57:58,  4.34s/it]
 26%|██▌       | 2562/10000 [3:02:23<9:01:58,  4.37s/it]
 26%|██▌       | 2563/10000 [3:02:28<9:14:57,  4.48s/it]
 26%|██▌       | 2564/10000 [3:02:32<9:03:20,  4.38s/it]
 26%|██▌       | 2565/10000 [3:02:36<8:55:58,  4.33s/it]
 26%|██▌       | 2566/10000 [3:02:40<9:01:36,  4.37s/it]
 26%|██▌       | 2567/10000 [3:02:45<8:54:31,  4.31s/it]
 26%|██▌       | 2568/10000 [3:02:49<9:00:18,  4.36s/it]
 26%|██▌       | 2569/10000 [3:02:53<8:55:05,  4.32s/it]
 26%|██▌       | 2570/10000 [3:02:58<8:54:27,  4.32s/it]
 26%|██▌       | 2571/10000 [3:03:02<9:09:41,  4.44s/it]
 26%|██▌       | 2572/10000 [3:03:06<9:00:28,  4.37s/it]
 26%|██▌       | 2573/10000 [3:03:11<9:04:24,  4.40s/it]
 26%|██▌       | 2574/10000 [3:03:15<8:58:11,  4.35s/it]
 26%|██▌       | 2575/10000 [3:03:19<8:52:01,  4.30s/it]
 26%|██▌       | 2576/10000 [3:03:24<9:00:11,  4.37s/it]
 26%|██▌       | 2577/10000 [3:03:28<9:04:30,  4.40s/it]
 26%|██▌       | 2578/10000 [3:03:33<8:57:48,  4.35s/it]
 26%|██▌       | 2579/10000 [3:03:37<9:06:38,  4.42s/it]
 26%|██▌       | 2580/10000 [3:03:41<8:58:44,  4.36s/it]
                                                        

 26%|██▌       | 2580/10000 [3:03:41<8:58:44,  4.36s/it]
 26%|██▌       | 2581/10000 [3:03:45<8:44:24,  4.24s/it]
 26%|██▌       | 2582/10000 [3:03:50<8:53:52,  4.32s/it]
 26%|██▌       | 2583/10000 [3:03:54<8:49:49,  4.29s/it]
 26%|██▌       | 2584/10000 [3:03:59<8:57:14,  4.35s/it]
 26%|██▌       | 2585/10000 [3:04:03<8:51:03,  4.30s/it]
 26%|██▌       | 2586/10000 [3:04:07<9:07:10,  4.43s/it]
 26%|██▌       | 2587/10000 [3:04:12<9:01:31,  4.38s/it]
 26%|██▌       | 2588/10000 [3:04:16<8:54:17,  4.33s/it]
 26%|██▌       | 2589/10000 [3:04:20<8:57:51,  4.35s/it]
 26%|██▌       | 2590/10000 [3:04:25<8:52:17,  4.31s/it]
 26%|██▌       | 2591/10000 [3:04:29<8:47:52,  4.27s/it]
 26%|██▌       | 2592/10000 [3:04:33<8:55:41,  4.34s/it]
 26%|██▌       | 2593/10000 [3:04:37<8:50:19,  4.30s/it]
 26%|██▌       | 2594/10000 [3:04:42<8:45:20,  4.26s/it]
 26%|██▌       | 2595/10000 [3:04:46<8:53:02,  4.32s/it]
 26%|██▌       | 2596/10000 [3:04:51<9:06:11,  4.43s/it]
 26%|██▌       | 2597/10000 [3:04:55<9:09:11,  4.45s/it]
 26%|██▌       | 2598/10000 [3:04:59<8:58:46,  4.37s/it]
 26%|██▌       | 2599/10000 [3:05:04<8:51:50,  4.31s/it]
 26%|██▌       | 2600/10000 [3:05:08<8:54:54,  4.34s/it]
                                                        

 26%|██▌       | 2600/10000 [3:05:08<8:54:54,  4.34s/it]
 26%|██▌       | 2601/10000 [3:05:12<8:49:08,  4.29s/it]
 26%|██▌       | 2602/10000 [3:05:16<8:45:09,  4.26s/it]
 26%|██▌       | 2603/10000 [3:05:21<8:53:08,  4.32s/it]
 26%|██▌       | 2604/10000 [3:05:25<8:55:58,  4.35s/it]
 26%|██▌       | 2605/10000 [3:05:30<9:00:06,  4.38s/it]
 26%|██▌       | 2606/10000 [3:05:34<8:53:15,  4.33s/it]
 26%|██▌       | 2607/10000 [3:05:38<8:49:29,  4.30s/it]
 26%|██▌       | 2608/10000 [3:05:43<8:57:18,  4.36s/it]
 26%|██▌       | 2609/10000 [3:05:47<8:52:38,  4.32s/it]
 26%|██▌       | 2610/10000 [3:05:51<8:48:23,  4.29s/it]
 26%|██▌       | 2611/10000 [3:05:56<8:55:37,  4.35s/it]
 26%|██▌       | 2612/10000 [3:06:00<8:50:26,  4.31s/it]
 26%|██▌       | 2613/10000 [3:06:04<8:46:47,  4.28s/it]
 26%|██▌       | 2614/10000 [3:06:09<9:01:15,  4.40s/it]
 26%|██▌       | 2615/10000 [3:06:13<8:54:11,  4.34s/it]
 26%|██▌       | 2616/10000 [3:06:17<8:59:56,  4.39s/it]
 26%|██▌       | 2617/10000 [3:06:22<8:53:17,  4.33s/it]
 26%|██▌       | 2618/10000 [3:06:26<8:49:04,  4.30s/it]
 26%|██▌       | 2619/10000 [3:06:30<8:47:24,  4.29s/it]
 26%|██▌       | 2620/10000 [3:06:34<8:43:16,  4.25s/it]
                                                        

 26%|██▌       | 2620/10000 [3:06:34<8:43:16,  4.25s/it]
 26%|██▌       | 2621/10000 [3:06:39<8:51:17,  4.32s/it]
 26%|██▌       | 2622/10000 [3:06:43<8:46:00,  4.28s/it]
 26%|██▌       | 2623/10000 [3:06:47<8:43:28,  4.26s/it]
 26%|██▌       | 2624/10000 [3:06:52<8:55:54,  4.36s/it]
 26%|██▋       | 2625/10000 [3:06:56<8:40:03,  4.23s/it]
 26%|██▋       | 2626/10000 [3:07:00<8:38:23,  4.22s/it]
 26%|██▋       | 2627/10000 [3:07:04<8:39:22,  4.23s/it]
 26%|██▋       | 2628/10000 [3:07:08<8:39:01,  4.22s/it]
 26%|██▋       | 2629/10000 [3:07:13<8:55:34,  4.36s/it]
 26%|██▋       | 2630/10000 [3:07:17<8:59:26,  4.39s/it]
 26%|██▋       | 2631/10000 [3:07:22<8:52:31,  4.34s/it]
 26%|██▋       | 2632/10000 [3:07:26<9:00:24,  4.40s/it]
 26%|██▋       | 2633/10000 [3:07:30<8:52:23,  4.34s/it]
 26%|██▋       | 2634/10000 [3:07:35<8:46:13,  4.29s/it]
 26%|██▋       | 2635/10000 [3:07:39<8:53:31,  4.35s/it]
 26%|██▋       | 2636/10000 [3:07:43<8:47:38,  4.30s/it]
 26%|██▋       | 2637/10000 [3:07:47<8:43:27,  4.27s/it]
 26%|██▋       | 2638/10000 [3:07:52<8:54:38,  4.36s/it]
 26%|██▋       | 2639/10000 [3:07:56<8:47:04,  4.30s/it]
 26%|██▋       | 2640/10000 [3:08:00<8:34:53,  4.20s/it]
                                                        

 26%|██▋       | 2640/10000 [3:08:00<8:34:53,  4.20s/it]
 26%|██▋       | 2641/10000 [3:08:05<8:44:59,  4.28s/it]
 26%|██▋       | 2642/10000 [3:08:09<8:41:44,  4.25s/it]
 26%|██▋       | 2643/10000 [3:08:13<8:49:48,  4.32s/it]
 26%|██▋       | 2644/10000 [3:08:17<8:46:04,  4.29s/it]
 26%|██▋       | 2645/10000 [3:08:22<8:41:52,  4.26s/it]
 26%|██▋       | 2646/10000 [3:08:26<8:41:58,  4.26s/it]
 26%|██▋       | 2647/10000 [3:08:30<8:40:01,  4.24s/it]
 26%|██▋       | 2648/10000 [3:08:35<8:50:50,  4.33s/it]
 26%|██▋       | 2649/10000 [3:08:39<8:46:34,  4.30s/it]
 26%|██▋       | 2650/10000 [3:08:43<8:45:43,  4.29s/it]
 27%|██▋       | 2651/10000 [3:08:48<8:53:08,  4.35s/it]
 27%|██▋       | 2652/10000 [3:08:52<8:47:40,  4.31s/it]
 27%|██▋       | 2653/10000 [3:08:56<8:43:57,  4.28s/it]
 27%|██▋       | 2654/10000 [3:09:01<8:52:05,  4.35s/it]
 27%|██▋       | 2655/10000 [3:09:05<8:51:50,  4.34s/it]
 27%|██▋       | 2656/10000 [3:09:10<9:10:21,  4.50s/it]
 27%|██▋       | 2657/10000 [3:09:14<8:59:12,  4.41s/it]
 27%|██▋       | 2658/10000 [3:09:18<8:43:32,  4.28s/it]
 27%|██▋       | 2659/10000 [3:09:22<8:52:19,  4.35s/it]
 27%|██▋       | 2660/10000 [3:09:27<8:46:26,  4.30s/it]
                                                        

 27%|██▋       | 2660/10000 [3:09:27<8:46:26,  4.30s/it]
 27%|██▋       | 2661/10000 [3:09:31<8:42:51,  4.27s/it]
 27%|██▋       | 2662/10000 [3:09:36<8:59:31,  4.41s/it]
 27%|██▋       | 2663/10000 [3:09:40<8:51:31,  4.35s/it]
 27%|██▋       | 2664/10000 [3:09:44<8:57:28,  4.40s/it]
 27%|██▋       | 2665/10000 [3:09:48<8:45:54,  4.30s/it]
 27%|██▋       | 2666/10000 [3:09:52<8:28:58,  4.16s/it]
 27%|██▋       | 2667/10000 [3:09:57<8:36:46,  4.23s/it]
 27%|██▋       | 2668/10000 [3:10:01<8:28:27,  4.16s/it]
 27%|██▋       | 2669/10000 [3:10:05<8:25:54,  4.14s/it]
 27%|██▋       | 2670/10000 [3:10:09<8:33:56,  4.21s/it]
 27%|██▋       | 2671/10000 [3:10:13<8:29:26,  4.17s/it]
 27%|██▋       | 2672/10000 [3:10:18<8:35:42,  4.22s/it]
 27%|██▋       | 2673/10000 [3:10:22<8:30:45,  4.18s/it]
 27%|██▋       | 2674/10000 [3:10:26<8:27:13,  4.15s/it]
 27%|██▋       | 2675/10000 [3:10:30<8:42:51,  4.28s/it]
 27%|██▋       | 2676/10000 [3:10:34<8:28:16,  4.16s/it]
 27%|██▋       | 2677/10000 [3:10:38<8:31:49,  4.19s/it]
 27%|██▋       | 2678/10000 [3:10:42<8:26:21,  4.15s/it]
 27%|██▋       | 2679/10000 [3:10:47<8:28:30,  4.17s/it]
 27%|██▋       | 2680/10000 [3:10:51<8:40:08,  4.26s/it]
                                                        

 27%|██▋       | 2680/10000 [3:10:51<8:40:08,  4.26s/it]
 27%|██▋       | 2681/10000 [3:10:56<8:44:33,  4.30s/it]
 27%|██▋       | 2682/10000 [3:11:00<8:42:23,  4.28s/it]
 27%|██▋       | 2683/10000 [3:11:04<8:38:42,  4.25s/it]
 27%|██▋       | 2684/10000 [3:11:08<8:36:10,  4.23s/it]
 27%|██▋       | 2685/10000 [3:11:13<8:45:31,  4.31s/it]
 27%|██▋       | 2686/10000 [3:11:17<8:46:24,  4.32s/it]
 27%|██▋       | 2687/10000 [3:11:21<8:41:40,  4.28s/it]
 27%|██▋       | 2688/10000 [3:11:26<8:49:12,  4.34s/it]
 27%|██▋       | 2689/10000 [3:11:30<8:41:18,  4.28s/it]
 27%|██▋       | 2690/10000 [3:11:34<8:56:00,  4.40s/it]
 27%|██▋       | 2691/10000 [3:11:39<8:46:06,  4.32s/it]
 27%|██▋       | 2692/10000 [3:11:43<8:41:46,  4.28s/it]
 27%|██▋       | 2693/10000 [3:11:47<8:48:48,  4.34s/it]
 27%|██▋       | 2694/10000 [3:11:51<8:43:16,  4.30s/it]
 27%|██▋       | 2695/10000 [3:11:56<8:49:39,  4.35s/it]
 27%|██▋       | 2696/10000 [3:12:00<8:44:10,  4.31s/it]
 27%|██▋       | 2697/10000 [3:12:04<8:40:32,  4.28s/it]
 27%|██▋       | 2698/10000 [3:12:09<9:08:01,  4.50s/it]
 27%|██▋       | 2699/10000 [3:12:14<8:55:31,  4.40s/it]
 27%|██▋       | 2700/10000 [3:12:18<8:48:01,  4.34s/it]
                                                        

 27%|██▋       | 2700/10000 [3:12:18<8:48:01,  4.34s/it]
 27%|██▋       | 2701/10000 [3:12:22<9:01:03,  4.45s/it]
 27%|██▋       | 2702/10000 [3:12:27<8:53:16,  4.38s/it]
 27%|██▋       | 2703/10000 [3:12:31<8:57:03,  4.42s/it]
 27%|██▋       | 2704/10000 [3:12:35<8:48:59,  4.35s/it]
 27%|██▋       | 2705/10000 [3:12:40<8:42:17,  4.30s/it]
 27%|██▋       | 2706/10000 [3:12:44<8:49:17,  4.35s/it]
 27%|██▋       | 2707/10000 [3:12:49<9:02:27,  4.46s/it]
 27%|██▋       | 2708/10000 [3:12:53<9:03:23,  4.47s/it]
 27%|██▋       | 2709/10000 [3:12:57<8:53:16,  4.39s/it]
 27%|██▋       | 2710/10000 [3:13:02<8:50:29,  4.37s/it]
 27%|██▋       | 2711/10000 [3:13:07<9:04:40,  4.48s/it]
 27%|██▋       | 2712/10000 [3:13:11<8:56:31,  4.42s/it]
 27%|██▋       | 2713/10000 [3:13:15<8:59:06,  4.44s/it]
 27%|██▋       | 2714/10000 [3:13:19<8:50:32,  4.37s/it]
 27%|██▋       | 2715/10000 [3:13:24<8:43:56,  4.32s/it]
 27%|██▋       | 2716/10000 [3:13:28<8:50:49,  4.37s/it]
 27%|██▋       | 2717/10000 [3:13:32<8:44:16,  4.32s/it]
 27%|██▋       | 2718/10000 [3:13:37<8:50:46,  4.37s/it]
 27%|██▋       | 2719/10000 [3:13:41<8:45:01,  4.33s/it]
 27%|██▋       | 2720/10000 [3:13:45<8:40:06,  4.29s/it]
                                                        
{'loss': '0.1102', 'grad_norm': '12.15', 'learning_rate': '3.731e-06', 'num_tokens': '7.521e+05', 'completions/mean_length': '4.144', 'completions/min_length': '3.65', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.144', 'completions/min_terminated_length': '3.65', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.397', 'rewards/reward_semantic/std': '0.08279', 'rewards/reward_length/mean': '-0.04221', 'rewards/reward_length/std': '0.01005', 'reward': '1.117', 'reward_std': '0.02508', 'frac_reward_zero_std': '0.5', 'kl': '2.789', 'entropy': '0.1916', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.283', 'epoch': '0.4167'}
{'loss': '0.1322', 'grad_norm': '0.0006197', 'learning_rate': '3.721e-06', 'num_tokens': '7.578e+05', 'completions/mean_length': '4.631', 'completions/min_length': '4.1', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.631', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2071', 'rewards/reward_semantic/std': '0.02542', 'rewards/reward_length/mean': '-0.03401', 'rewards/reward_length/std': '0.008759', 'reward': '1.06', 'reward_std': '0.007939', 'frac_reward_zero_std': '0.6', 'kl': '2.795', 'entropy': '0.2657', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.335', 'epoch': '0.4199'}
{'loss': '0.06426', 'grad_norm': '0.05361', 'learning_rate': '3.711e-06', 'num_tokens': '7.637e+05', 'completions/mean_length': '4.794', 'completions/min_length': '4.05', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.794', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.231', 'rewards/reward_semantic/std': '0.05088', 'rewards/reward_length/mean': '-0.02898', 'rewards/reward_length/std': '0.01285', 'reward': '1.068', 'reward_std': '0.01505', 'frac_reward_zero_std': '0.5', 'kl': '2.46', 'entropy': '0.2191', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.34', 'epoch': '0.4232'}
{'loss': '0.1403', 'grad_norm': '0.001501', 'learning_rate': '3.701e-06', 'num_tokens': '7.696e+05', 'completions/mean_length': '4.7', 'completions/min_length': '4.5', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.7', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3458', 'rewards/reward_semantic/std': '0.007715', 'rewards/reward_length/mean': '-0.02646', 'rewards/reward_length/std': '0.002194', 'reward': '1.102', 'reward_std': '0.002424', 'frac_reward_zero_std': '0.85', 'kl': '2.856', 'entropy': '0.07091', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.311', 'epoch': '0.4265'}
{'loss': '0.1087', 'grad_norm': '0.01398', 'learning_rate': '3.691e-06', 'num_tokens': '7.756e+05', 'completions/mean_length': '4.831', 'completions/min_length': '4.6', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.831', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1463', 'rewards/reward_semantic/std': '0.03695', 'rewards/reward_length/mean': '-0.01992', 'rewards/reward_length/std': '0.002126', 'reward': '1.043', 'reward_std': '0.0111', 'frac_reward_zero_std': '0.85', 'kl': '3.034', 'entropy': '0.05088', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.292', 'epoch': '0.4298'}
{'loss': '0.1542', 'grad_norm': '0.0009662', 'learning_rate': '3.681e-06', 'num_tokens': '7.818e+05', 'completions/mean_length': '4.469', 'completions/min_length': '4.05', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3754', 'rewards/reward_semantic/std': '0.06737', 'rewards/reward_length/mean': '-0.02358', 'rewards/reward_length/std': '0.003268', 'reward': '1.111', 'reward_std': '0.02006', 'frac_reward_zero_std': '0.75', 'kl': '3.172', 'entropy': '0.1175', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.273', 'epoch': '0.4331'}
{'loss': '0.08397', 'grad_norm': '0.7024', 'learning_rate': '3.671e-06', 'num_tokens': '7.877e+05', 'completions/mean_length': '4.612', 'completions/min_length': '4.3', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.612', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2058', 'rewards/reward_semantic/std': '0.04492', 'rewards/reward_length/mean': '-0.02375', 'rewards/reward_length/std': '0.007752', 'reward': '1.061', 'reward_std': '0.0134', 'frac_reward_zero_std': '0.65', 'kl': '3.048', 'entropy': '0.1454', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.307', 'epoch': '0.4364'}
{'loss': '0.1433', 'grad_norm': '0.1156', 'learning_rate': '3.661e-06', 'num_tokens': '7.937e+05', 'completions/mean_length': '4.513', 'completions/min_length': '4.35', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.513', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4338', 'rewards/reward_semantic/std': '0.03975', 'rewards/reward_length/mean': '-0.02747', 'rewards/reward_length/std': '0.003061', 'reward': '1.129', 'reward_std': '0.01186', 'frac_reward_zero_std': '0.8', 'kl': '3.435', 'entropy': '0.07575', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.206', 'epoch': '0.4396'}
{'loss': '0.08729', 'grad_norm': '0.08691', 'learning_rate': '3.651e-06', 'num_tokens': '7.996e+05', 'completions/mean_length': '4.706', 'completions/min_length': '4.4', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.706', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3838', 'rewards/reward_semantic/std': '0.01703', 'rewards/reward_length/mean': '-0.0301', 'rewards/reward_length/std': '0.007275', 'reward': '1.114', 'reward_std': '0.005185', 'frac_reward_zero_std': '0.75', 'kl': '2.8', 'entropy': '0.08705', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.31', 'epoch': '0.4429'}

 27%|██▋       | 2720/10000 [3:13:45<8:40:06,  4.29s/it]
 27%|██▋       | 2721/10000 [3:13:50<8:54:58,  4.41s/it]
 27%|██▋       | 2722/10000 [3:13:54<8:47:23,  4.35s/it]
 27%|██▋       | 2723/10000 [3:13:59<8:59:55,  4.45s/it]
 27%|██▋       | 2724/10000 [3:14:03<8:53:28,  4.40s/it]
 27%|██▋       | 2725/10000 [3:14:08<8:55:58,  4.42s/it]
 27%|██▋       | 2726/10000 [3:14:12<8:59:04,  4.45s/it]
 27%|██▋       | 2727/10000 [3:14:16<8:49:28,  4.37s/it]
 27%|██▋       | 2728/10000 [3:14:21<8:51:52,  4.39s/it]
 27%|██▋       | 2729/10000 [3:14:25<8:42:21,  4.31s/it]
 27%|██▋       | 2730/10000 [3:14:29<8:48:57,  4.37s/it]
 27%|██▋       | 2731/10000 [3:14:34<9:01:21,  4.47s/it]
 27%|██▋       | 2732/10000 [3:14:38<8:51:52,  4.39s/it]
 27%|██▋       | 2733/10000 [3:14:43<8:56:06,  4.43s/it]
 27%|██▋       | 2734/10000 [3:14:47<8:48:21,  4.36s/it]
 27%|██▋       | 2735/10000 [3:14:51<8:42:51,  4.32s/it]
 27%|██▋       | 2736/10000 [3:14:56<8:49:45,  4.38s/it]
 27%|██▋       | 2737/10000 [3:15:00<8:44:02,  4.33s/it]
 27%|██▋       | 2738/10000 [3:15:04<8:51:06,  4.39s/it]
 27%|██▋       | 2739/10000 [3:15:09<9:01:05,  4.47s/it]
 27%|██▋       | 2740/10000 [3:15:14<9:02:33,  4.48s/it]
                                                        

 27%|██▋       | 2740/10000 [3:15:14<9:02:33,  4.48s/it]
 27%|██▋       | 2741/10000 [3:15:18<8:44:29,  4.34s/it]
 27%|██▋       | 2742/10000 [3:15:22<8:40:23,  4.30s/it]
 27%|██▋       | 2743/10000 [3:15:26<8:51:27,  4.39s/it]
 27%|██▋       | 2744/10000 [3:15:31<8:57:14,  4.44s/it]
 27%|██▋       | 2745/10000 [3:15:36<8:58:25,  4.45s/it]
 27%|██▋       | 2746/10000 [3:15:40<8:51:32,  4.40s/it]
 27%|██▋       | 2747/10000 [3:15:44<8:55:12,  4.43s/it]
 27%|██▋       | 2748/10000 [3:15:48<8:47:22,  4.36s/it]
 27%|██▋       | 2749/10000 [3:15:52<8:33:15,  4.25s/it]
 28%|██▊       | 2750/10000 [3:15:57<8:39:59,  4.30s/it]
 28%|██▊       | 2751/10000 [3:16:01<8:36:47,  4.28s/it]
 28%|██▊       | 2752/10000 [3:16:06<8:45:14,  4.35s/it]
 28%|██▊       | 2753/10000 [3:16:10<8:40:31,  4.31s/it]
 28%|██▊       | 2754/10000 [3:16:14<8:39:03,  4.30s/it]
 28%|██▊       | 2755/10000 [3:16:19<8:46:24,  4.36s/it]
 28%|██▊       | 2756/10000 [3:16:23<8:40:41,  4.31s/it]
 28%|██▊       | 2757/10000 [3:16:27<8:47:30,  4.37s/it]
 28%|██▊       | 2758/10000 [3:16:32<8:42:15,  4.33s/it]
 28%|██▊       | 2759/10000 [3:16:36<8:53:43,  4.42s/it]
 28%|██▊       | 2760/10000 [3:16:40<8:37:29,  4.29s/it]
                                                        

 28%|██▊       | 2760/10000 [3:16:40<8:37:29,  4.29s/it]
 28%|██▊       | 2761/10000 [3:16:44<8:37:33,  4.29s/it]
 28%|██▊       | 2762/10000 [3:16:49<8:46:11,  4.36s/it]
 28%|██▊       | 2763/10000 [3:16:53<8:41:11,  4.32s/it]
 28%|██▊       | 2764/10000 [3:16:58<8:49:52,  4.39s/it]
 28%|██▊       | 2765/10000 [3:17:02<8:43:14,  4.34s/it]
 28%|██▊       | 2766/10000 [3:17:06<8:39:03,  4.31s/it]
 28%|██▊       | 2767/10000 [3:17:11<8:46:32,  4.37s/it]
 28%|██▊       | 2768/10000 [3:17:15<8:42:18,  4.33s/it]
 28%|██▊       | 2769/10000 [3:17:20<8:50:02,  4.40s/it]
 28%|██▊       | 2770/10000 [3:17:24<8:44:06,  4.35s/it]
 28%|██▊       | 2771/10000 [3:17:28<8:49:38,  4.40s/it]
 28%|██▊       | 2772/10000 [3:17:32<8:43:06,  4.34s/it]
 28%|██▊       | 2773/10000 [3:17:37<8:38:10,  4.30s/it]
 28%|██▊       | 2774/10000 [3:17:41<8:44:32,  4.36s/it]
 28%|██▊       | 2775/10000 [3:17:45<8:31:20,  4.25s/it]
 28%|██▊       | 2776/10000 [3:17:50<8:41:15,  4.33s/it]
 28%|██▊       | 2777/10000 [3:17:54<8:37:42,  4.30s/it]
 28%|██▊       | 2778/10000 [3:17:58<8:45:45,  4.37s/it]
 28%|██▊       | 2779/10000 [3:18:03<8:40:54,  4.33s/it]
 28%|██▊       | 2780/10000 [3:18:07<8:27:24,  4.22s/it]
                                                        

 28%|██▊       | 2780/10000 [3:18:07<8:27:24,  4.22s/it]
 28%|██▊       | 2781/10000 [3:18:11<8:38:51,  4.31s/it]
 28%|██▊       | 2782/10000 [3:18:15<8:27:04,  4.22s/it]
 28%|██▊       | 2783/10000 [3:18:20<8:38:03,  4.31s/it]
 28%|██▊       | 2784/10000 [3:18:24<8:27:01,  4.22s/it]
 28%|██▊       | 2785/10000 [3:18:28<8:29:31,  4.24s/it]
 28%|██▊       | 2786/10000 [3:18:33<8:45:34,  4.37s/it]
 28%|██▊       | 2787/10000 [3:18:37<8:30:45,  4.25s/it]
 28%|██▊       | 2788/10000 [3:18:41<8:48:02,  4.39s/it]
 28%|██▊       | 2789/10000 [3:18:46<8:40:15,  4.33s/it]
 28%|██▊       | 2790/10000 [3:18:50<8:45:26,  4.37s/it]
 28%|██▊       | 2791/10000 [3:18:54<8:38:42,  4.32s/it]
 28%|██▊       | 2792/10000 [3:18:58<8:33:28,  4.27s/it]
 28%|██▊       | 2793/10000 [3:19:03<8:44:33,  4.37s/it]
 28%|██▊       | 2794/10000 [3:19:07<8:37:37,  4.31s/it]
 28%|██▊       | 2795/10000 [3:19:11<8:34:52,  4.29s/it]
 28%|██▊       | 2796/10000 [3:19:16<8:33:54,  4.28s/it]
 28%|██▊       | 2797/10000 [3:19:20<8:22:21,  4.18s/it]
 28%|██▊       | 2798/10000 [3:19:24<8:29:58,  4.25s/it]
 28%|██▊       | 2799/10000 [3:19:28<8:27:38,  4.23s/it]
 28%|██▊       | 2800/10000 [3:19:33<8:37:18,  4.31s/it]
                                                        

 28%|██▊       | 2800/10000 [3:19:33<8:37:18,  4.31s/it]
 28%|██▊       | 2801/10000 [3:19:37<8:31:56,  4.27s/it]
 28%|██▊       | 2802/10000 [3:19:41<8:41:20,  4.35s/it]
 28%|██▊       | 2803/10000 [3:19:46<8:35:18,  4.30s/it]
 28%|██▊       | 2804/10000 [3:19:50<8:30:09,  4.25s/it]
 28%|██▊       | 2805/10000 [3:19:54<8:38:10,  4.32s/it]
 28%|██▊       | 2806/10000 [3:19:58<8:32:57,  4.28s/it]
 28%|██▊       | 2807/10000 [3:20:03<8:40:22,  4.34s/it]
 28%|██▊       | 2808/10000 [3:20:07<8:27:09,  4.23s/it]
 28%|██▊       | 2809/10000 [3:20:11<8:27:02,  4.23s/it]
 28%|██▊       | 2810/10000 [3:20:16<8:35:55,  4.31s/it]
 28%|██▊       | 2811/10000 [3:20:20<8:34:34,  4.29s/it]
 28%|██▊       | 2812/10000 [3:20:24<8:44:12,  4.38s/it]
 28%|██▊       | 2813/10000 [3:20:29<8:36:56,  4.32s/it]
 28%|██▊       | 2814/10000 [3:20:33<8:43:47,  4.37s/it]
 28%|██▊       | 2815/10000 [3:20:37<8:38:07,  4.33s/it]
 28%|██▊       | 2816/10000 [3:20:41<8:30:08,  4.26s/it]
 28%|██▊       | 2817/10000 [3:20:46<8:40:18,  4.35s/it]
 28%|██▊       | 2818/10000 [3:20:50<8:34:35,  4.30s/it]
 28%|██▊       | 2819/10000 [3:20:55<8:41:01,  4.35s/it]
 28%|██▊       | 2820/10000 [3:20:59<8:35:13,  4.31s/it]
                                                        

 28%|██▊       | 2820/10000 [3:20:59<8:35:13,  4.31s/it]
 28%|██▊       | 2821/10000 [3:21:03<8:30:11,  4.26s/it]
 28%|██▊       | 2822/10000 [3:21:08<8:42:15,  4.37s/it]
 28%|██▊       | 2823/10000 [3:21:12<8:34:51,  4.30s/it]
 28%|██▊       | 2824/10000 [3:21:16<8:38:38,  4.34s/it]
 28%|██▊       | 2825/10000 [3:21:20<8:27:30,  4.24s/it]
 28%|██▊       | 2826/10000 [3:21:24<8:20:25,  4.19s/it]
 28%|██▊       | 2827/10000 [3:21:29<8:31:02,  4.27s/it]
 28%|██▊       | 2828/10000 [3:21:33<8:29:48,  4.26s/it]
 28%|██▊       | 2829/10000 [3:21:38<8:41:58,  4.37s/it]
 28%|██▊       | 2830/10000 [3:21:42<8:35:19,  4.31s/it]
 28%|██▊       | 2831/10000 [3:21:46<8:41:36,  4.37s/it]
 28%|██▊       | 2832/10000 [3:21:50<8:34:28,  4.31s/it]
 28%|██▊       | 2833/10000 [3:21:55<8:30:46,  4.28s/it]
 28%|██▊       | 2834/10000 [3:21:59<8:38:25,  4.34s/it]
 28%|██▊       | 2835/10000 [3:22:03<8:25:42,  4.23s/it]
 28%|██▊       | 2836/10000 [3:22:08<8:35:46,  4.32s/it]
 28%|██▊       | 2837/10000 [3:22:12<8:31:49,  4.29s/it]
 28%|██▊       | 2838/10000 [3:22:16<8:38:43,  4.35s/it]
 28%|██▊       | 2839/10000 [3:22:21<8:43:14,  4.38s/it]
 28%|██▊       | 2840/10000 [3:22:25<8:36:42,  4.33s/it]
                                                        

 28%|██▊       | 2840/10000 [3:22:25<8:36:42,  4.33s/it]
 28%|██▊       | 2841/10000 [3:22:29<8:42:15,  4.38s/it]
 28%|██▊       | 2842/10000 [3:22:34<8:32:47,  4.30s/it]
 28%|██▊       | 2843/10000 [3:22:38<8:35:41,  4.32s/it]
 28%|██▊       | 2844/10000 [3:22:42<8:27:20,  4.25s/it]
 28%|██▊       | 2845/10000 [3:22:46<8:32:12,  4.30s/it]
 28%|██▊       | 2846/10000 [3:22:51<8:31:06,  4.29s/it]
 28%|██▊       | 2847/10000 [3:22:55<8:25:32,  4.24s/it]
 28%|██▊       | 2848/10000 [3:22:59<8:31:56,  4.29s/it]
 28%|██▊       | 2849/10000 [3:23:03<8:25:31,  4.24s/it]
 28%|██▊       | 2850/10000 [3:23:08<8:28:40,  4.27s/it]
 29%|██▊       | 2851/10000 [3:23:12<8:23:56,  4.23s/it]
 29%|██▊       | 2852/10000 [3:23:16<8:31:48,  4.30s/it]
 29%|██▊       | 2853/10000 [3:23:20<8:25:38,  4.24s/it]
 29%|██▊       | 2854/10000 [3:23:25<8:22:33,  4.22s/it]
 29%|██▊       | 2855/10000 [3:23:29<8:29:24,  4.28s/it]
 29%|██▊       | 2856/10000 [3:23:33<8:29:01,  4.28s/it]
 29%|██▊       | 2857/10000 [3:23:38<8:35:36,  4.33s/it]
 29%|██▊       | 2858/10000 [3:23:42<8:28:35,  4.27s/it]
 29%|██▊       | 2859/10000 [3:23:46<8:22:30,  4.22s/it]
 29%|██▊       | 2860/10000 [3:23:50<8:32:13,  4.30s/it]
                                                        

 29%|██▊       | 2860/10000 [3:23:50<8:32:13,  4.30s/it]
 29%|██▊       | 2861/10000 [3:23:55<8:29:16,  4.28s/it]
 29%|██▊       | 2862/10000 [3:23:59<8:37:11,  4.35s/it]
 29%|██▊       | 2863/10000 [3:24:04<8:36:21,  4.34s/it]
 29%|██▊       | 2864/10000 [3:24:08<8:41:53,  4.39s/it]
 29%|██▊       | 2865/10000 [3:24:13<8:50:50,  4.46s/it]
 29%|██▊       | 2866/10000 [3:24:17<8:42:23,  4.39s/it]
 29%|██▊       | 2867/10000 [3:24:21<8:45:40,  4.42s/it]
 29%|██▊       | 2868/10000 [3:24:26<8:38:16,  4.36s/it]
 29%|██▊       | 2869/10000 [3:24:30<8:40:10,  4.38s/it]
 29%|██▊       | 2870/10000 [3:24:34<8:35:58,  4.34s/it]
 29%|██▊       | 2871/10000 [3:24:39<8:41:54,  4.39s/it]
 29%|██▊       | 2872/10000 [3:24:43<8:35:16,  4.34s/it]
 29%|██▊       | 2873/10000 [3:24:47<8:30:19,  4.30s/it]
 29%|██▊       | 2874/10000 [3:24:52<8:36:25,  4.35s/it]
 29%|██▉       | 2875/10000 [3:24:56<8:48:22,  4.45s/it]
 29%|██▉       | 2876/10000 [3:25:01<8:49:46,  4.46s/it]
 29%|██▉       | 2877/10000 [3:25:05<8:35:13,  4.34s/it]
 29%|██▉       | 2878/10000 [3:25:09<8:40:39,  4.39s/it]
 29%|██▉       | 2879/10000 [3:25:14<8:37:22,  4.36s/it]
 29%|██▉       | 2880/10000 [3:25:18<8:34:25,  4.34s/it]
                                                        

 29%|██▉       | 2880/10000 [3:25:18<8:34:25,  4.34s/it]
 29%|██▉       | 2881/10000 [3:25:22<8:40:18,  4.39s/it]
 29%|██▉       | 2882/10000 [3:25:26<8:26:26,  4.27s/it]
 29%|██▉       | 2883/10000 [3:25:31<8:34:58,  4.34s/it]
 29%|██▉       | 2884/10000 [3:25:35<8:25:39,  4.26s/it]
 29%|██▉       | 2885/10000 [3:25:39<8:21:09,  4.23s/it]
 29%|██▉       | 2886/10000 [3:25:44<8:34:24,  4.34s/it]
 29%|██▉       | 2887/10000 [3:25:48<8:45:46,  4.44s/it]
 29%|██▉       | 2888/10000 [3:25:53<8:52:44,  4.49s/it]
 29%|██▉       | 2889/10000 [3:25:57<8:33:34,  4.33s/it]
 29%|██▉       | 2890/10000 [3:26:02<8:41:10,  4.40s/it]
 29%|██▉       | 2891/10000 [3:26:06<8:34:30,  4.34s/it]
 29%|██▉       | 2892/10000 [3:26:10<8:30:31,  4.31s/it]
 29%|██▉       | 2893/10000 [3:26:15<8:37:20,  4.37s/it]
 29%|██▉       | 2894/10000 [3:26:18<8:23:15,  4.25s/it]
 29%|██▉       | 2895/10000 [3:26:23<8:32:44,  4.33s/it]
 29%|██▉       | 2896/10000 [3:26:27<8:29:29,  4.30s/it]
 29%|██▉       | 2897/10000 [3:26:32<8:41:19,  4.40s/it]
 29%|██▉       | 2898/10000 [3:26:36<8:44:14,  4.43s/it]
 29%|██▉       | 2899/10000 [3:26:41<8:36:34,  4.36s/it]
 29%|██▉       | 2900/10000 [3:26:45<8:46:58,  4.45s/it]
                                                        
{'loss': '0.1219', 'grad_norm': '0.004686', 'learning_rate': '3.641e-06', 'num_tokens': '8.055e+05', 'completions/mean_length': '5.025', 'completions/min_length': '4.65', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.025', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1501', 'rewards/reward_semantic/std': '0.03324', 'rewards/reward_length/mean': '-0.02932', 'rewards/reward_length/std': '0.004861', 'reward': '1.044', 'reward_std': '0.0101', 'frac_reward_zero_std': '0.65', 'kl': '3.14', 'entropy': '0.1072', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.356', 'epoch': '0.4462'}
{'loss': '0.0908', 'grad_norm': '9.513e-05', 'learning_rate': '3.631e-06', 'num_tokens': '8.115e+05', 'completions/mean_length': '4.969', 'completions/min_length': '4.45', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.969', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2468', 'rewards/reward_semantic/std': '0.01602', 'rewards/reward_length/mean': '-0.03169', 'rewards/reward_length/std': '0.006785', 'reward': '1.072', 'reward_std': '0.004886', 'frac_reward_zero_std': '0.6', 'kl': '2.546', 'entropy': '0.1931', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.399', 'epoch': '0.4495'}
{'loss': '0.19', 'grad_norm': '0.005968', 'learning_rate': '3.621e-06', 'num_tokens': '8.174e+05', 'completions/mean_length': '4.1', 'completions/min_length': '3.9', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.1', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4008', 'rewards/reward_semantic/std': '0.1044', 'rewards/reward_length/mean': '-0.02227', 'rewards/reward_length/std': '0.005004', 'reward': '1.119', 'reward_std': '0.03136', 'frac_reward_zero_std': '0.65', 'kl': '3.078', 'entropy': '0.1515', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.306', 'epoch': '0.4528'}
{'loss': '0.1097', 'grad_norm': '0.01469', 'learning_rate': '3.61e-06', 'num_tokens': '8.233e+05', 'completions/mean_length': '4.638', 'completions/min_length': '4.35', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.638', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3937', 'rewards/reward_semantic/std': '0.01768', 'rewards/reward_length/mean': '-0.02261', 'rewards/reward_length/std': '0.002024', 'reward': '1.117', 'reward_std': '0.005211', 'frac_reward_zero_std': '0.9', 'kl': '3.219', 'entropy': '0.04595', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.304', 'epoch': '0.456'}
{'loss': '0.08936', 'grad_norm': '10.93', 'learning_rate': '3.601e-06', 'num_tokens': '8.289e+05', 'completions/mean_length': '3.9', 'completions/min_length': '3.6', 'completions/max_length': '4.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '3.9', 'completions/min_terminated_length': '3.6', 'completions/max_terminated_length': '4.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5042', 'rewards/reward_semantic/std': '0.02946', 'rewards/reward_length/mean': '-0.02822', 'rewards/reward_length/std': '0.005533', 'reward': '1.15', 'reward_std': '0.008855', 'frac_reward_zero_std': '0.65', 'kl': '3.069', 'entropy': '0.2142', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.282', 'epoch': '0.4593'}
{'loss': '0.07297', 'grad_norm': '0.0001202', 'learning_rate': '3.591e-06', 'num_tokens': '8.351e+05', 'completions/mean_length': '4.469', 'completions/min_length': '4.1', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2659', 'rewards/reward_semantic/std': '0.04579', 'rewards/reward_length/mean': '-0.02563', 'rewards/reward_length/std': '0.005602', 'reward': '1.078', 'reward_std': '0.01374', 'frac_reward_zero_std': '0.7', 'kl': '2.837', 'entropy': '0.1234', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.285', 'epoch': '0.4626'}
{'loss': '0.07162', 'grad_norm': '0.0272', 'learning_rate': '3.581e-06', 'num_tokens': '8.408e+05', 'completions/mean_length': '4.25', 'completions/min_length': '3.9', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.25', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2737', 'rewards/reward_semantic/std': '0.03977', 'rewards/reward_length/mean': '-0.02888', 'rewards/reward_length/std': '0.006691', 'reward': '1.081', 'reward_std': '0.01184', 'frac_reward_zero_std': '0.65', 'kl': '3.152', 'entropy': '0.1469', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.288', 'epoch': '0.4659'}
{'loss': '0.09849', 'grad_norm': '0.0001588', 'learning_rate': '3.57e-06', 'num_tokens': '8.468e+05', 'completions/mean_length': '4.812', 'completions/min_length': '4.65', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.812', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.192', 'rewards/reward_semantic/std': '0.04082', 'rewards/reward_length/mean': '-0.02454', 'rewards/reward_length/std': '0.003883', 'reward': '1.056', 'reward_std': '0.0122', 'frac_reward_zero_std': '0.85', 'kl': '2.865', 'entropy': '0.06931', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.256', 'epoch': '0.4692'}
{'loss': '0.07551', 'grad_norm': '7.663', 'learning_rate': '3.561e-06', 'num_tokens': '8.529e+05', 'completions/mean_length': '4.869', 'completions/min_length': '4.25', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.869', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1478', 'rewards/reward_semantic/std': '0.07768', 'rewards/reward_length/mean': '-0.03394', 'rewards/reward_length/std': '0.009229', 'reward': '1.043', 'reward_std': '0.02309', 'frac_reward_zero_std': '0.65', 'kl': '2.558', 'entropy': '0.1537', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.354', 'epoch': '0.4724'}

 29%|██▉       | 2900/10000 [3:26:45<8:46:58,  4.45s/it]
 29%|██▉       | 2901/10000 [3:26:49<8:38:21,  4.38s/it]
 29%|██▉       | 2902/10000 [3:26:54<8:42:55,  4.42s/it]
 29%|██▉       | 2903/10000 [3:26:58<8:35:42,  4.36s/it]
 29%|██▉       | 2904/10000 [3:27:02<8:22:52,  4.25s/it]
 29%|██▉       | 2905/10000 [3:27:07<8:30:30,  4.32s/it]
 29%|██▉       | 2906/10000 [3:27:11<8:31:38,  4.33s/it]
 29%|██▉       | 2907/10000 [3:27:16<8:37:53,  4.38s/it]
 29%|██▉       | 2908/10000 [3:27:20<8:35:59,  4.37s/it]
 29%|██▉       | 2909/10000 [3:27:24<8:41:20,  4.41s/it]
 29%|██▉       | 2910/10000 [3:27:29<8:33:45,  4.35s/it]
 29%|██▉       | 2911/10000 [3:27:33<8:28:31,  4.30s/it]
 29%|██▉       | 2912/10000 [3:27:37<8:32:26,  4.34s/it]
 29%|██▉       | 2913/10000 [3:27:41<8:26:59,  4.29s/it]
 29%|██▉       | 2914/10000 [3:27:46<8:36:26,  4.37s/it]
 29%|██▉       | 2915/10000 [3:27:50<8:30:02,  4.32s/it]
 29%|██▉       | 2916/10000 [3:27:55<8:38:06,  4.39s/it]
 29%|██▉       | 2917/10000 [3:27:59<8:31:12,  4.33s/it]
 29%|██▉       | 2918/10000 [3:28:03<8:19:58,  4.24s/it]
 29%|██▉       | 2919/10000 [3:28:07<8:29:44,  4.32s/it]
 29%|██▉       | 2920/10000 [3:28:12<8:27:26,  4.30s/it]
                                                        

 29%|██▉       | 2920/10000 [3:28:12<8:27:26,  4.30s/it]
 29%|██▉       | 2921/10000 [3:28:16<8:35:36,  4.37s/it]
 29%|██▉       | 2922/10000 [3:28:21<8:43:29,  4.44s/it]
 29%|██▉       | 2923/10000 [3:28:25<8:42:05,  4.43s/it]
 29%|██▉       | 2924/10000 [3:28:30<8:45:22,  4.45s/it]
 29%|██▉       | 2925/10000 [3:28:34<8:37:26,  4.39s/it]
 29%|██▉       | 2926/10000 [3:28:38<8:42:22,  4.43s/it]
 29%|██▉       | 2927/10000 [3:28:43<8:35:21,  4.37s/it]
 29%|██▉       | 2928/10000 [3:28:47<8:40:37,  4.42s/it]
 29%|██▉       | 2929/10000 [3:28:51<8:33:38,  4.36s/it]
 29%|██▉       | 2930/10000 [3:28:56<8:29:01,  4.32s/it]
 29%|██▉       | 2931/10000 [3:29:00<8:35:48,  4.38s/it]
 29%|██▉       | 2932/10000 [3:29:04<8:30:34,  4.33s/it]
 29%|██▉       | 2933/10000 [3:29:09<8:37:05,  4.39s/it]
 29%|██▉       | 2934/10000 [3:29:13<8:31:17,  4.34s/it]
 29%|██▉       | 2935/10000 [3:29:17<8:20:00,  4.25s/it]
 29%|██▉       | 2936/10000 [3:29:22<8:30:22,  4.34s/it]
 29%|██▉       | 2937/10000 [3:29:26<8:27:12,  4.31s/it]
 29%|██▉       | 2938/10000 [3:29:31<8:40:34,  4.42s/it]
 29%|██▉       | 2939/10000 [3:29:35<8:37:17,  4.40s/it]
 29%|██▉       | 2940/10000 [3:29:40<8:53:18,  4.53s/it]
                                                        

 29%|██▉       | 2940/10000 [3:29:40<8:53:18,  4.53s/it]
 29%|██▉       | 2941/10000 [3:29:44<8:43:22,  4.45s/it]
 29%|██▉       | 2942/10000 [3:29:48<8:40:19,  4.42s/it]
 29%|██▉       | 2943/10000 [3:29:53<8:35:45,  4.39s/it]
 29%|██▉       | 2944/10000 [3:29:57<8:30:06,  4.34s/it]
 29%|██▉       | 2945/10000 [3:30:01<8:28:47,  4.33s/it]
 29%|██▉       | 2946/10000 [3:30:05<8:18:03,  4.24s/it]
 29%|██▉       | 2947/10000 [3:30:10<8:32:16,  4.36s/it]
 29%|██▉       | 2948/10000 [3:30:14<8:28:06,  4.32s/it]
 29%|██▉       | 2949/10000 [3:30:18<8:25:29,  4.30s/it]
 30%|██▉       | 2950/10000 [3:30:23<8:29:11,  4.33s/it]
 30%|██▉       | 2951/10000 [3:30:27<8:28:25,  4.33s/it]
 30%|██▉       | 2952/10000 [3:30:32<8:35:59,  4.39s/it]
 30%|██▉       | 2953/10000 [3:30:36<8:30:51,  4.35s/it]
 30%|██▉       | 2954/10000 [3:30:41<8:54:29,  4.55s/it]
 30%|██▉       | 2955/10000 [3:30:45<8:43:53,  4.46s/it]
 30%|██▉       | 2956/10000 [3:30:50<8:40:58,  4.44s/it]
 30%|██▉       | 2957/10000 [3:30:55<8:56:56,  4.57s/it]
 30%|██▉       | 2958/10000 [3:30:59<8:36:56,  4.40s/it]
 30%|██▉       | 2959/10000 [3:31:03<8:40:31,  4.44s/it]
 30%|██▉       | 2960/10000 [3:31:07<8:38:50,  4.42s/it]
                                                        

 30%|██▉       | 2960/10000 [3:31:07<8:38:50,  4.42s/it]
 30%|██▉       | 2961/10000 [3:31:12<8:26:39,  4.32s/it]
 30%|██▉       | 2962/10000 [3:31:16<8:33:30,  4.38s/it]
 30%|██▉       | 2963/10000 [3:31:21<8:43:48,  4.47s/it]
 30%|██▉       | 2964/10000 [3:31:25<8:45:40,  4.48s/it]
 30%|██▉       | 2965/10000 [3:31:29<8:36:05,  4.40s/it]
 30%|██▉       | 2966/10000 [3:31:34<8:39:50,  4.43s/it]
 30%|██▉       | 2967/10000 [3:31:38<8:30:59,  4.36s/it]
 30%|██▉       | 2968/10000 [3:31:42<8:26:23,  4.32s/it]
 30%|██▉       | 2969/10000 [3:31:47<8:24:03,  4.30s/it]
 30%|██▉       | 2970/10000 [3:31:51<8:33:29,  4.38s/it]
 30%|██▉       | 2971/10000 [3:31:56<8:37:56,  4.42s/it]
 30%|██▉       | 2972/10000 [3:32:00<8:30:30,  4.36s/it]
 30%|██▉       | 2973/10000 [3:32:04<8:36:11,  4.41s/it]
 30%|██▉       | 2974/10000 [3:32:09<8:28:57,  4.35s/it]
 30%|██▉       | 2975/10000 [3:32:13<8:16:03,  4.24s/it]
 30%|██▉       | 2976/10000 [3:32:17<8:25:04,  4.31s/it]
 30%|██▉       | 2977/10000 [3:32:21<8:18:25,  4.26s/it]
 30%|██▉       | 2978/10000 [3:32:26<8:26:23,  4.33s/it]
 30%|██▉       | 2979/10000 [3:32:30<8:21:44,  4.29s/it]
 30%|██▉       | 2980/10000 [3:32:34<8:18:10,  4.26s/it]
                                                        

 30%|██▉       | 2980/10000 [3:32:34<8:18:10,  4.26s/it]
 30%|██▉       | 2981/10000 [3:32:39<8:26:06,  4.33s/it]
 30%|██▉       | 2982/10000 [3:32:43<8:23:21,  4.30s/it]
 30%|██▉       | 2983/10000 [3:32:47<8:28:45,  4.35s/it]
 30%|██▉       | 2984/10000 [3:32:51<8:18:34,  4.26s/it]
 30%|██▉       | 2985/10000 [3:32:56<8:26:19,  4.33s/it]
 30%|██▉       | 2986/10000 [3:33:00<8:36:34,  4.42s/it]
 30%|██▉       | 2987/10000 [3:33:05<8:28:39,  4.35s/it]
 30%|██▉       | 2988/10000 [3:33:10<8:46:12,  4.50s/it]
 30%|██▉       | 2989/10000 [3:33:14<8:35:55,  4.42s/it]
 30%|██▉       | 2990/10000 [3:33:19<9:02:42,  4.65s/it]
 30%|██▉       | 2991/10000 [3:33:23<8:47:06,  4.51s/it]
 30%|██▉       | 2992/10000 [3:33:28<8:46:52,  4.51s/it]
 30%|██▉       | 2993/10000 [3:33:32<8:35:58,  4.42s/it]
 30%|██▉       | 2994/10000 [3:33:36<8:29:13,  4.36s/it]
 30%|██▉       | 2995/10000 [3:33:40<8:29:54,  4.37s/it]
 30%|██▉       | 2996/10000 [3:33:44<8:11:48,  4.21s/it]
 30%|██▉       | 2997/10000 [3:33:48<8:09:43,  4.20s/it]
 30%|██▉       | 2998/10000 [3:33:53<8:15:33,  4.25s/it]
 30%|██▉       | 2999/10000 [3:33:57<8:26:38,  4.34s/it]
 30%|███       | 3000/10000 [3:34:01<8:17:34,  4.26s/it]
                                                        

 30%|███       | 3000/10000 [3:34:01<8:17:34,  4.26s/it]
 30%|███       | 3001/10000 [3:34:07<8:47:59,  4.53s/it]
 30%|███       | 3002/10000 [3:34:11<8:46:00,  4.51s/it]
 30%|███       | 3003/10000 [3:34:15<8:32:15,  4.39s/it]
 30%|███       | 3004/10000 [3:34:20<8:36:18,  4.43s/it]
 30%|███       | 3005/10000 [3:34:24<8:24:38,  4.33s/it]
 30%|███       | 3006/10000 [3:34:28<8:18:37,  4.28s/it]
 30%|███       | 3007/10000 [3:34:32<8:12:41,  4.23s/it]
 30%|███       | 3008/10000 [3:34:36<8:07:42,  4.19s/it]
 30%|███       | 3009/10000 [3:34:41<8:27:58,  4.36s/it]
 30%|███       | 3010/10000 [3:34:45<8:18:30,  4.28s/it]
 30%|███       | 3011/10000 [3:34:49<8:21:46,  4.31s/it]
 30%|███       | 3012/10000 [3:34:54<8:17:16,  4.27s/it]
 30%|███       | 3013/10000 [3:34:58<8:10:11,  4.21s/it]
 30%|███       | 3014/10000 [3:35:02<8:15:14,  4.25s/it]
 30%|███       | 3015/10000 [3:35:06<8:18:53,  4.29s/it]
 30%|███       | 3016/10000 [3:35:11<8:22:13,  4.31s/it]
 30%|███       | 3017/10000 [3:35:15<8:23:43,  4.33s/it]
 30%|███       | 3018/10000 [3:35:19<8:25:28,  4.34s/it]
 30%|███       | 3019/10000 [3:35:24<8:16:26,  4.27s/it]
 30%|███       | 3020/10000 [3:35:27<8:01:51,  4.14s/it]
                                                        

 30%|███       | 3020/10000 [3:35:27<8:01:51,  4.14s/it]
 30%|███       | 3021/10000 [3:35:32<8:17:05,  4.27s/it]
 30%|███       | 3022/10000 [3:35:36<8:05:25,  4.17s/it]
 30%|███       | 3023/10000 [3:35:40<8:12:28,  4.24s/it]
 30%|███       | 3024/10000 [3:35:44<8:08:26,  4.20s/it]
 30%|███       | 3025/10000 [3:35:49<8:15:32,  4.26s/it]
 30%|███       | 3026/10000 [3:35:53<8:10:40,  4.22s/it]
 30%|███       | 3027/10000 [3:35:57<8:04:31,  4.17s/it]
 30%|███       | 3028/10000 [3:36:02<8:27:13,  4.37s/it]
 30%|███       | 3029/10000 [3:36:06<8:18:28,  4.29s/it]
 30%|███       | 3030/10000 [3:36:11<8:32:11,  4.41s/it]
 30%|███       | 3031/10000 [3:36:15<8:29:30,  4.39s/it]
 30%|███       | 3032/10000 [3:36:19<8:30:55,  4.40s/it]
 30%|███       | 3033/10000 [3:36:24<8:20:36,  4.31s/it]
 30%|███       | 3034/10000 [3:36:28<8:13:35,  4.25s/it]
 30%|███       | 3035/10000 [3:36:32<8:19:04,  4.30s/it]
 30%|███       | 3036/10000 [3:36:36<8:04:37,  4.18s/it]
 30%|███       | 3037/10000 [3:36:40<8:12:47,  4.25s/it]
 30%|███       | 3038/10000 [3:36:44<8:08:21,  4.21s/it]
 30%|███       | 3039/10000 [3:36:49<8:05:28,  4.18s/it]
 30%|███       | 3040/10000 [3:36:53<8:13:40,  4.26s/it]
                                                        

 30%|███       | 3040/10000 [3:36:53<8:13:40,  4.26s/it]
 30%|███       | 3041/10000 [3:36:57<8:08:54,  4.22s/it]
 30%|███       | 3042/10000 [3:37:02<8:15:27,  4.27s/it]
 30%|███       | 3043/10000 [3:37:06<8:12:38,  4.25s/it]
 30%|███       | 3044/10000 [3:37:10<8:24:12,  4.35s/it]
 30%|███       | 3045/10000 [3:37:15<8:28:19,  4.39s/it]
 30%|███       | 3046/10000 [3:37:19<8:19:27,  4.31s/it]
 30%|███       | 3047/10000 [3:37:23<8:22:46,  4.34s/it]
 30%|███       | 3048/10000 [3:37:27<8:15:03,  4.27s/it]
 30%|███       | 3049/10000 [3:37:32<8:11:37,  4.24s/it]
 30%|███       | 3050/10000 [3:37:36<8:11:43,  4.25s/it]
 31%|███       | 3051/10000 [3:37:40<8:17:31,  4.30s/it]
 31%|███       | 3052/10000 [3:37:44<8:12:08,  4.25s/it]
 31%|███       | 3053/10000 [3:37:49<8:07:27,  4.21s/it]
 31%|███       | 3054/10000 [3:37:53<8:19:01,  4.31s/it]
 31%|███       | 3055/10000 [3:37:57<8:11:56,  4.25s/it]
 31%|███       | 3056/10000 [3:38:02<8:17:33,  4.30s/it]
 31%|███       | 3057/10000 [3:38:06<8:12:22,  4.25s/it]
 31%|███       | 3058/10000 [3:38:10<8:17:37,  4.30s/it]
 31%|███       | 3059/10000 [3:38:14<8:03:51,  4.18s/it]
 31%|███       | 3060/10000 [3:38:19<8:18:06,  4.31s/it]
                                                        

 31%|███       | 3060/10000 [3:38:19<8:18:06,  4.31s/it]
 31%|███       | 3061/10000 [3:38:23<8:25:02,  4.37s/it]
 31%|███       | 3062/10000 [3:38:28<8:24:23,  4.36s/it]
 31%|███       | 3063/10000 [3:38:32<8:20:59,  4.33s/it]
 31%|███       | 3064/10000 [3:38:36<8:26:42,  4.38s/it]
 31%|███       | 3065/10000 [3:38:41<8:28:13,  4.40s/it]
 31%|███       | 3066/10000 [3:38:45<8:21:37,  4.34s/it]
 31%|███       | 3067/10000 [3:38:49<8:17:02,  4.30s/it]
 31%|███       | 3068/10000 [3:38:53<8:16:10,  4.29s/it]
 31%|███       | 3069/10000 [3:38:57<8:05:09,  4.20s/it]
 31%|███       | 3070/10000 [3:39:02<8:15:16,  4.29s/it]
 31%|███       | 3071/10000 [3:39:06<8:12:26,  4.26s/it]
 31%|███       | 3072/10000 [3:39:11<8:32:44,  4.44s/it]
 31%|███       | 3073/10000 [3:39:15<8:25:12,  4.38s/it]
 31%|███       | 3074/10000 [3:39:19<8:18:39,  4.32s/it]
 31%|███       | 3075/10000 [3:39:24<8:30:29,  4.42s/it]
 31%|███       | 3076/10000 [3:39:28<8:23:11,  4.36s/it]
 31%|███       | 3077/10000 [3:39:33<8:19:51,  4.33s/it]
 31%|███       | 3078/10000 [3:39:37<8:13:00,  4.27s/it]
 31%|███       | 3079/10000 [3:39:41<8:16:25,  4.30s/it]
 31%|███       | 3080/10000 [3:39:45<8:07:49,  4.23s/it]
                                                        
{'loss': '0.1173', 'grad_norm': '15.48', 'learning_rate': '3.551e-06', 'num_tokens': '8.585e+05', 'completions/mean_length': '4.106', 'completions/min_length': '3.55', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.106', 'completions/min_terminated_length': '3.55', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.436', 'rewards/reward_semantic/std': '0.07995', 'rewards/reward_length/mean': '-0.03726', 'rewards/reward_length/std': '0.01232', 'reward': '1.129', 'reward_std': '0.02399', 'frac_reward_zero_std': '0.55', 'kl': '2.725', 'entropy': '0.2005', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.346', 'epoch': '0.4757'}
{'loss': '0.11', 'grad_norm': '13.15', 'learning_rate': '3.541e-06', 'num_tokens': '8.645e+05', 'completions/mean_length': '4.438', 'completions/min_length': '4.25', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.438', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1966', 'rewards/reward_semantic/std': '0.02756', 'rewards/reward_length/mean': '-0.03181', 'rewards/reward_length/std': '0.003496', 'reward': '1.057', 'reward_std': '0.008444', 'frac_reward_zero_std': '0.65', 'kl': '2.883', 'entropy': '0.1683', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.3', 'epoch': '0.479'}
{'loss': '0.07886', 'grad_norm': '0.02443', 'learning_rate': '3.53e-06', 'num_tokens': '8.706e+05', 'completions/mean_length': '4.869', 'completions/min_length': '4.55', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.869', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4701', 'rewards/reward_semantic/std': '0.01364', 'rewards/reward_length/mean': '-0.02659', 'rewards/reward_length/std': '0.003322', 'reward': '1.14', 'reward_std': '0.004073', 'frac_reward_zero_std': '0.8', 'kl': '2.547', 'entropy': '0.1393', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.39', 'epoch': '0.4823'}
{'loss': '0.1479', 'grad_norm': '8.774', 'learning_rate': '3.521e-06', 'num_tokens': '8.765e+05', 'completions/mean_length': '4.513', 'completions/min_length': '4', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.513', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4019', 'rewards/reward_semantic/std': '0.06777', 'rewards/reward_length/mean': '-0.0396', 'rewards/reward_length/std': '0.005959', 'reward': '1.119', 'reward_std': '0.02039', 'frac_reward_zero_std': '0.55', 'kl': '2.62', 'entropy': '0.1913', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.359', 'epoch': '0.4856'}
{'loss': '0.09841', 'grad_norm': '0.005356', 'learning_rate': '3.511e-06', 'num_tokens': '8.825e+05', 'completions/mean_length': '4.869', 'completions/min_length': '4.7', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.869', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3202', 'rewards/reward_semantic/std': '0.01277', 'rewards/reward_length/mean': '-0.02454', 'rewards/reward_length/std': '0.00111', 'reward': '1.095', 'reward_std': '0.003782', 'frac_reward_zero_std': '0.9', 'kl': '2.823', 'entropy': '0.04637', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.314', 'epoch': '0.4888'}
{'loss': '0.06671', 'grad_norm': '0.01266', 'learning_rate': '3.501e-06', 'num_tokens': '8.885e+05', 'completions/mean_length': '4.662', 'completions/min_length': '4.1', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.662', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3322', 'rewards/reward_semantic/std': '0.07411', 'rewards/reward_length/mean': '-0.02908', 'rewards/reward_length/std': '0.01226', 'reward': '1.098', 'reward_std': '0.02192', 'frac_reward_zero_std': '0.55', 'kl': '2.902', 'entropy': '0.1784', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.348', 'epoch': '0.4921'}
{'loss': '0.1228', 'grad_norm': '0.03647', 'learning_rate': '3.491e-06', 'num_tokens': '8.945e+05', 'completions/mean_length': '4.675', 'completions/min_length': '4.1', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.675', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2538', 'rewards/reward_semantic/std': '0.03641', 'rewards/reward_length/mean': '-0.02969', 'rewards/reward_length/std': '0.008483', 'reward': '1.075', 'reward_std': '0.01084', 'frac_reward_zero_std': '0.55', 'kl': '2.897', 'entropy': '0.2559', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.223', 'epoch': '0.4954'}
{'loss': '0.08569', 'grad_norm': '8.778', 'learning_rate': '3.481e-06', 'num_tokens': '9.003e+05', 'completions/mean_length': '4.781', 'completions/min_length': '4.55', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.781', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3016', 'rewards/reward_semantic/std': '0.04464', 'rewards/reward_length/mean': '-0.03091', 'rewards/reward_length/std': '0.008532', 'reward': '1.089', 'reward_std': '0.01331', 'frac_reward_zero_std': '0.7', 'kl': '2.652', 'entropy': '0.1565', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.261', 'epoch': '0.4987'}
{'loss': '0.1079', 'grad_norm': '3.449', 'learning_rate': '3.471e-06', 'num_tokens': '9.062e+05', 'completions/mean_length': '4.781', 'completions/min_length': '4.4', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.781', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5105', 'rewards/reward_semantic/std': '0.1035', 'rewards/reward_length/mean': '-0.03088', 'rewards/reward_length/std': '0.008213', 'reward': '1.152', 'reward_std': '0.03086', 'frac_reward_zero_std': '0.65', 'kl': '2.692', 'entropy': '0.164', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.265', 'epoch': '0.502'}

 31%|███       | 3080/10000 [3:39:45<8:07:49,  4.23s/it]
 31%|███       | 3081/10000 [3:39:49<8:14:06,  4.28s/it]
 31%|███       | 3082/10000 [3:39:54<8:07:13,  4.23s/it]
 31%|███       | 3083/10000 [3:39:58<8:13:42,  4.28s/it]
 31%|███       | 3084/10000 [3:40:02<8:06:42,  4.22s/it]
 31%|███       | 3085/10000 [3:40:06<8:02:24,  4.19s/it]
 31%|███       | 3086/10000 [3:40:11<8:09:41,  4.25s/it]
 31%|███       | 3087/10000 [3:40:15<8:04:15,  4.20s/it]
 31%|███       | 3088/10000 [3:40:19<8:19:29,  4.34s/it]
 31%|███       | 3089/10000 [3:40:23<8:11:17,  4.27s/it]
 31%|███       | 3090/10000 [3:40:28<8:08:37,  4.24s/it]
 31%|███       | 3091/10000 [3:40:32<8:06:02,  4.22s/it]
 31%|███       | 3092/10000 [3:40:36<8:10:19,  4.26s/it]
 31%|███       | 3093/10000 [3:40:40<8:07:30,  4.23s/it]
 31%|███       | 3094/10000 [3:40:45<8:24:41,  4.38s/it]
 31%|███       | 3095/10000 [3:40:50<8:30:51,  4.44s/it]
 31%|███       | 3096/10000 [3:40:54<8:29:13,  4.43s/it]
 31%|███       | 3097/10000 [3:40:58<8:15:20,  4.31s/it]
 31%|███       | 3098/10000 [3:41:02<8:18:12,  4.33s/it]
 31%|███       | 3099/10000 [3:41:07<8:09:54,  4.26s/it]
 31%|███       | 3100/10000 [3:41:11<8:18:11,  4.33s/it]
                                                        

 31%|███       | 3100/10000 [3:41:11<8:18:11,  4.33s/it]
 31%|███       | 3101/10000 [3:41:16<8:24:09,  4.38s/it]
 31%|███       | 3102/10000 [3:41:20<8:22:48,  4.37s/it]
 31%|███       | 3103/10000 [3:41:24<8:12:42,  4.29s/it]
 31%|███       | 3104/10000 [3:41:28<8:06:29,  4.23s/it]
 31%|███       | 3105/10000 [3:41:33<8:13:54,  4.30s/it]
 31%|███       | 3106/10000 [3:41:37<8:26:23,  4.41s/it]
 31%|███       | 3107/10000 [3:41:42<8:25:28,  4.40s/it]
 31%|███       | 3108/10000 [3:41:46<8:19:31,  4.35s/it]
 31%|███       | 3109/10000 [3:41:50<8:21:48,  4.37s/it]
 31%|███       | 3110/10000 [3:41:55<8:22:56,  4.38s/it]
 31%|███       | 3111/10000 [3:41:59<8:27:28,  4.42s/it]
 31%|███       | 3112/10000 [3:42:03<8:16:01,  4.32s/it]
 31%|███       | 3113/10000 [3:42:08<8:16:22,  4.32s/it]
 31%|███       | 3114/10000 [3:42:12<8:09:23,  4.26s/it]
 31%|███       | 3115/10000 [3:42:17<8:34:36,  4.48s/it]
 31%|███       | 3116/10000 [3:42:21<8:22:18,  4.38s/it]
 31%|███       | 3117/10000 [3:42:25<8:12:53,  4.30s/it]
 31%|███       | 3118/10000 [3:42:29<8:15:48,  4.32s/it]
 31%|███       | 3119/10000 [3:42:33<8:08:36,  4.26s/it]
 31%|███       | 3120/10000 [3:42:38<8:14:10,  4.31s/it]
                                                        

 31%|███       | 3120/10000 [3:42:38<8:14:10,  4.31s/it]
 31%|███       | 3121/10000 [3:42:42<8:15:14,  4.32s/it]
 31%|███       | 3122/10000 [3:42:46<8:10:24,  4.28s/it]
 31%|███       | 3123/10000 [3:42:51<8:09:26,  4.27s/it]
 31%|███       | 3124/10000 [3:42:55<8:16:09,  4.33s/it]
 31%|███▏      | 3125/10000 [3:42:59<8:18:17,  4.35s/it]
 31%|███▏      | 3126/10000 [3:43:04<8:23:44,  4.40s/it]
 31%|███▏      | 3127/10000 [3:43:08<8:13:23,  4.31s/it]
 31%|███▏      | 3128/10000 [3:43:13<8:25:43,  4.42s/it]
 31%|███▏      | 3129/10000 [3:43:17<8:14:32,  4.32s/it]
 31%|███▏      | 3130/10000 [3:43:22<8:35:01,  4.50s/it]
 31%|███▏      | 3131/10000 [3:43:26<8:23:37,  4.40s/it]
 31%|███▏      | 3132/10000 [3:43:30<8:24:59,  4.41s/it]
 31%|███▏      | 3133/10000 [3:43:34<8:11:18,  4.29s/it]
 31%|███▏      | 3134/10000 [3:43:38<8:05:49,  4.25s/it]
 31%|███▏      | 3135/10000 [3:43:43<8:24:07,  4.41s/it]
 31%|███▏      | 3136/10000 [3:43:47<8:14:15,  4.32s/it]
 31%|███▏      | 3137/10000 [3:43:52<8:20:25,  4.38s/it]
 31%|███▏      | 3138/10000 [3:43:56<8:08:21,  4.27s/it]
 31%|███▏      | 3139/10000 [3:44:00<8:05:17,  4.24s/it]
 31%|███▏      | 3140/10000 [3:44:05<8:12:06,  4.30s/it]
                                                        

 31%|███▏      | 3140/10000 [3:44:05<8:12:06,  4.30s/it]
 31%|███▏      | 3141/10000 [3:44:09<8:01:25,  4.21s/it]
 31%|███▏      | 3142/10000 [3:44:13<8:13:31,  4.32s/it]
 31%|███▏      | 3143/10000 [3:44:17<8:11:26,  4.30s/it]
 31%|███▏      | 3144/10000 [3:44:22<8:29:10,  4.46s/it]
 31%|███▏      | 3145/10000 [3:44:27<8:24:00,  4.41s/it]
 31%|███▏      | 3146/10000 [3:44:31<8:25:43,  4.43s/it]
 31%|███▏      | 3147/10000 [3:44:35<8:22:23,  4.40s/it]
 31%|███▏      | 3148/10000 [3:44:40<8:28:17,  4.45s/it]
 31%|███▏      | 3149/10000 [3:44:44<8:20:15,  4.38s/it]
 32%|███▏      | 3150/10000 [3:44:49<8:31:14,  4.48s/it]
 32%|███▏      | 3151/10000 [3:44:53<8:14:24,  4.33s/it]
 32%|███▏      | 3152/10000 [3:44:57<8:25:46,  4.43s/it]
 32%|███▏      | 3153/10000 [3:45:02<8:18:46,  4.37s/it]
 32%|███▏      | 3154/10000 [3:45:06<8:30:48,  4.48s/it]
 32%|███▏      | 3155/10000 [3:45:11<8:22:27,  4.40s/it]
 32%|███▏      | 3156/10000 [3:45:15<8:26:35,  4.44s/it]
 32%|███▏      | 3157/10000 [3:45:20<8:28:45,  4.46s/it]
 32%|███▏      | 3158/10000 [3:45:24<8:33:59,  4.51s/it]
 32%|███▏      | 3159/10000 [3:45:29<8:29:03,  4.46s/it]
 32%|███▏      | 3160/10000 [3:45:33<8:31:30,  4.49s/it]
                                                        

 32%|███▏      | 3160/10000 [3:45:33<8:31:30,  4.49s/it]
 32%|███▏      | 3161/10000 [3:45:37<8:23:30,  4.42s/it]
 32%|███▏      | 3162/10000 [3:45:42<8:32:03,  4.49s/it]
 32%|███▏      | 3163/10000 [3:45:47<8:44:59,  4.61s/it]
 32%|███▏      | 3164/10000 [3:45:52<8:50:13,  4.65s/it]
 32%|███▏      | 3165/10000 [3:45:56<8:34:04,  4.51s/it]
 32%|███▏      | 3166/10000 [3:46:00<8:24:45,  4.43s/it]
 32%|███▏      | 3167/10000 [3:46:05<8:45:59,  4.62s/it]
 32%|███▏      | 3168/10000 [3:46:09<8:31:30,  4.49s/it]
 32%|███▏      | 3169/10000 [3:46:14<8:26:24,  4.45s/it]
 32%|███▏      | 3170/10000 [3:46:18<8:18:09,  4.38s/it]
 32%|███▏      | 3171/10000 [3:46:22<8:22:40,  4.42s/it]
 32%|███▏      | 3172/10000 [3:46:26<8:07:35,  4.28s/it]
 32%|███▏      | 3173/10000 [3:46:31<8:15:02,  4.35s/it]
 32%|███▏      | 3174/10000 [3:46:35<8:19:49,  4.39s/it]
 32%|███▏      | 3175/10000 [3:46:40<8:22:18,  4.42s/it]
 32%|███▏      | 3176/10000 [3:46:44<8:15:27,  4.36s/it]
 32%|███▏      | 3177/10000 [3:46:49<8:21:03,  4.41s/it]
 32%|███▏      | 3178/10000 [3:46:53<8:15:09,  4.35s/it]
 32%|███▏      | 3179/10000 [3:46:57<8:13:47,  4.34s/it]
 32%|███▏      | 3180/10000 [3:47:02<8:16:57,  4.37s/it]
                                                        

 32%|███▏      | 3180/10000 [3:47:02<8:16:57,  4.37s/it]
 32%|███▏      | 3181/10000 [3:47:06<8:21:09,  4.41s/it]
 32%|███▏      | 3182/10000 [3:47:10<8:18:13,  4.38s/it]
 32%|███▏      | 3183/10000 [3:47:14<8:04:58,  4.27s/it]
 32%|███▏      | 3184/10000 [3:47:19<8:09:21,  4.31s/it]
 32%|███▏      | 3185/10000 [3:47:23<8:05:35,  4.28s/it]
 32%|███▏      | 3186/10000 [3:47:28<8:14:47,  4.36s/it]
 32%|███▏      | 3187/10000 [3:47:32<8:12:16,  4.34s/it]
 32%|███▏      | 3188/10000 [3:47:36<8:19:04,  4.40s/it]
 32%|███▏      | 3189/10000 [3:47:41<8:11:29,  4.33s/it]
 32%|███▏      | 3190/10000 [3:47:45<8:21:49,  4.42s/it]
 32%|███▏      | 3191/10000 [3:47:50<8:26:56,  4.47s/it]
 32%|███▏      | 3192/10000 [3:47:54<8:23:16,  4.44s/it]
 32%|███▏      | 3193/10000 [3:47:59<8:27:20,  4.47s/it]
 32%|███▏      | 3194/10000 [3:48:03<8:18:52,  4.40s/it]
 32%|███▏      | 3195/10000 [3:48:08<8:24:44,  4.45s/it]
 32%|███▏      | 3196/10000 [3:48:12<8:17:23,  4.39s/it]
 32%|███▏      | 3197/10000 [3:48:16<8:23:29,  4.44s/it]
 32%|███▏      | 3198/10000 [3:48:21<8:18:23,  4.40s/it]
 32%|███▏      | 3199/10000 [3:48:25<8:23:37,  4.44s/it]
 32%|███▏      | 3200/10000 [3:48:29<8:17:32,  4.39s/it]
                                                        

 32%|███▏      | 3200/10000 [3:48:29<8:17:32,  4.39s/it]
 32%|███▏      | 3201/10000 [3:48:34<8:13:23,  4.35s/it]
 32%|███▏      | 3202/10000 [3:48:38<8:17:39,  4.39s/it]
 32%|███▏      | 3203/10000 [3:48:43<8:20:23,  4.42s/it]
 32%|███▏      | 3204/10000 [3:48:48<8:41:28,  4.60s/it]
 32%|███▏      | 3205/10000 [3:48:52<8:30:54,  4.51s/it]
 32%|███▏      | 3206/10000 [3:48:57<8:30:37,  4.51s/it]
 32%|███▏      | 3207/10000 [3:49:01<8:33:03,  4.53s/it]
 32%|███▏      | 3208/10000 [3:49:06<8:33:29,  4.54s/it]
 32%|███▏      | 3209/10000 [3:49:10<8:21:03,  4.43s/it]
 32%|███▏      | 3210/10000 [3:49:14<8:24:39,  4.46s/it]
 32%|███▏      | 3211/10000 [3:49:19<8:17:44,  4.40s/it]
 32%|███▏      | 3212/10000 [3:49:23<8:23:41,  4.45s/it]
 32%|███▏      | 3213/10000 [3:49:27<8:18:05,  4.40s/it]
 32%|███▏      | 3214/10000 [3:49:32<8:22:22,  4.44s/it]
 32%|███▏      | 3215/10000 [3:49:36<8:14:07,  4.37s/it]
 32%|███▏      | 3216/10000 [3:49:41<8:20:02,  4.42s/it]
 32%|███▏      | 3217/10000 [3:49:45<8:14:04,  4.37s/it]
 32%|███▏      | 3218/10000 [3:49:50<8:24:23,  4.46s/it]
 32%|███▏      | 3219/10000 [3:49:54<8:16:59,  4.40s/it]
 32%|███▏      | 3220/10000 [3:49:59<8:22:37,  4.45s/it]
                                                        

 32%|███▏      | 3220/10000 [3:49:59<8:22:37,  4.45s/it]
 32%|███▏      | 3221/10000 [3:50:03<8:16:30,  4.39s/it]
 32%|███▏      | 3222/10000 [3:50:07<8:20:50,  4.43s/it]
 32%|███▏      | 3223/10000 [3:50:12<8:17:34,  4.41s/it]
 32%|███▏      | 3224/10000 [3:50:16<8:14:56,  4.38s/it]
 32%|███▏      | 3225/10000 [3:50:21<8:21:02,  4.44s/it]
 32%|███▏      | 3226/10000 [3:50:25<8:22:31,  4.45s/it]
 32%|███▏      | 3227/10000 [3:50:30<8:24:43,  4.47s/it]
 32%|███▏      | 3228/10000 [3:50:34<8:17:00,  4.40s/it]
 32%|███▏      | 3229/10000 [3:50:38<8:18:01,  4.41s/it]
 32%|███▏      | 3230/10000 [3:50:42<8:07:29,  4.32s/it]
 32%|███▏      | 3231/10000 [3:50:47<8:02:22,  4.28s/it]
 32%|███▏      | 3232/10000 [3:50:51<8:11:13,  4.35s/it]
 32%|███▏      | 3233/10000 [3:50:55<8:07:34,  4.32s/it]
 32%|███▏      | 3234/10000 [3:51:00<8:12:01,  4.36s/it]
 32%|███▏      | 3235/10000 [3:51:04<8:13:14,  4.37s/it]
 32%|███▏      | 3236/10000 [3:51:09<8:21:23,  4.45s/it]
 32%|███▏      | 3237/10000 [3:51:13<8:08:49,  4.34s/it]
 32%|███▏      | 3238/10000 [3:51:17<8:15:42,  4.40s/it]
 32%|███▏      | 3239/10000 [3:51:22<8:10:58,  4.36s/it]
 32%|███▏      | 3240/10000 [3:51:27<8:29:29,  4.52s/it]
                                                        

 32%|███▏      | 3240/10000 [3:51:27<8:29:29,  4.52s/it]
 32%|███▏      | 3241/10000 [3:51:31<8:31:02,  4.54s/it]
 32%|███▏      | 3242/10000 [3:51:35<8:20:45,  4.45s/it]
 32%|███▏      | 3243/10000 [3:51:40<8:24:38,  4.48s/it]
 32%|███▏      | 3244/10000 [3:51:44<8:17:30,  4.42s/it]
 32%|███▏      | 3245/10000 [3:51:49<8:16:30,  4.41s/it]
 32%|███▏      | 3246/10000 [3:51:53<8:09:57,  4.35s/it]
 32%|███▏      | 3247/10000 [3:51:57<8:18:20,  4.43s/it]
 32%|███▏      | 3248/10000 [3:52:02<8:13:53,  4.39s/it]
 32%|███▏      | 3249/10000 [3:52:06<8:18:56,  4.43s/it]
 32%|███▎      | 3250/10000 [3:52:10<8:12:22,  4.38s/it]
 33%|███▎      | 3251/10000 [3:52:15<8:27:22,  4.51s/it]
 33%|███▎      | 3252/10000 [3:52:19<8:15:13,  4.40s/it]
 33%|███▎      | 3253/10000 [3:52:24<8:10:21,  4.36s/it]
 33%|███▎      | 3254/10000 [3:52:28<8:16:37,  4.42s/it]
 33%|███▎      | 3255/10000 [3:52:33<8:10:24,  4.36s/it]
 33%|███▎      | 3256/10000 [3:52:37<8:15:53,  4.41s/it]
 33%|███▎      | 3257/10000 [3:52:41<8:09:43,  4.36s/it]
 33%|███▎      | 3258/10000 [3:52:46<8:15:05,  4.41s/it]
 33%|███▎      | 3259/10000 [3:52:50<8:09:18,  4.36s/it]
 33%|███▎      | 3260/10000 [3:52:54<8:07:33,  4.34s/it]
                                                        
{'loss': '0.1475', 'grad_norm': '3.641e-05', 'learning_rate': '3.461e-06', 'num_tokens': '9.119e+05', 'completions/mean_length': '4.281', 'completions/min_length': '4.05', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.281', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4425', 'rewards/reward_semantic/std': '0.07365', 'rewards/reward_length/mean': '-0.03562', 'rewards/reward_length/std': '0.008338', 'reward': '1.131', 'reward_std': '0.02213', 'frac_reward_zero_std': '0.6', 'kl': '2.8', 'entropy': '0.168', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.301', 'epoch': '0.5052'}
{'loss': '0.09076', 'grad_norm': '10.66', 'learning_rate': '3.451e-06', 'num_tokens': '9.181e+05', 'completions/mean_length': '5.125', 'completions/min_length': '4.55', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.125', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4778', 'rewards/reward_semantic/std': '0.03208', 'rewards/reward_length/mean': '-0.03618', 'rewards/reward_length/std': '0.004346', 'reward': '1.142', 'reward_std': '0.009652', 'frac_reward_zero_std': '0.7', 'kl': '2.475', 'entropy': '0.1343', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.278', 'epoch': '0.5085'}
{'loss': '0.1225', 'grad_norm': '0.4195', 'learning_rate': '3.441e-06', 'num_tokens': '9.241e+05', 'completions/mean_length': '5.125', 'completions/min_length': '4.45', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.125', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3302', 'rewards/reward_semantic/std': '0.03472', 'rewards/reward_length/mean': '-0.03608', 'rewards/reward_length/std': '0.005601', 'reward': '1.097', 'reward_std': '0.01061', 'frac_reward_zero_std': '0.65', 'kl': '2.634', 'entropy': '0.2048', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.322', 'epoch': '0.5118'}
{'loss': '0.1103', 'grad_norm': '13.12', 'learning_rate': '3.431e-06', 'num_tokens': '9.299e+05', 'completions/mean_length': '4.344', 'completions/min_length': '3.25', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.344', 'completions/min_terminated_length': '3.25', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2252', 'rewards/reward_semantic/std': '0.09423', 'rewards/reward_length/mean': '-0.03655', 'rewards/reward_length/std': '0.01073', 'reward': '1.066', 'reward_std': '0.0282', 'frac_reward_zero_std': '0.3', 'kl': '2.881', 'entropy': '0.3388', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.319', 'epoch': '0.5151'}
{'loss': '0.09594', 'grad_norm': '9.709', 'learning_rate': '3.421e-06', 'num_tokens': '9.358e+05', 'completions/mean_length': '4.325', 'completions/min_length': '3.75', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.325', 'completions/min_terminated_length': '3.75', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2369', 'rewards/reward_semantic/std': '0.03953', 'rewards/reward_length/mean': '-0.02996', 'rewards/reward_length/std': '0.01085', 'reward': '1.07', 'reward_std': '0.01224', 'frac_reward_zero_std': '0.45', 'kl': '2.564', 'entropy': '0.3658', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.411', 'epoch': '0.5184'}
{'loss': '0.1113', 'grad_norm': '6.455', 'learning_rate': '3.411e-06', 'num_tokens': '9.418e+05', 'completions/mean_length': '4.888', 'completions/min_length': '4.45', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.888', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3786', 'rewards/reward_semantic/std': '0.06206', 'rewards/reward_length/mean': '-0.03806', 'rewards/reward_length/std': '0.009158', 'reward': '1.112', 'reward_std': '0.01876', 'frac_reward_zero_std': '0.65', 'kl': '2.949', 'entropy': '0.1936', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.404', 'epoch': '0.5217'}
{'loss': '0.09025', 'grad_norm': '0.06213', 'learning_rate': '3.401e-06', 'num_tokens': '9.478e+05', 'completions/mean_length': '4.65', 'completions/min_length': '4.4', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.65', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3863', 'rewards/reward_semantic/std': '0.08723', 'rewards/reward_length/mean': '-0.02988', 'rewards/reward_length/std': '0.007078', 'reward': '1.114', 'reward_std': '0.02605', 'frac_reward_zero_std': '0.75', 'kl': '2.745', 'entropy': '0.1398', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.37', 'epoch': '0.5249'}
{'loss': '0.1565', 'grad_norm': '0.006394', 'learning_rate': '3.391e-06', 'num_tokens': '9.539e+05', 'completions/mean_length': '4.819', 'completions/min_length': '4.15', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.819', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3702', 'rewards/reward_semantic/std': '0.09432', 'rewards/reward_length/mean': '-0.02844', 'rewards/reward_length/std': '0.009679', 'reward': '1.11', 'reward_std': '0.02838', 'frac_reward_zero_std': '0.6', 'kl': '2.682', 'entropy': '0.1547', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.432', 'epoch': '0.5282'}
{'loss': '0.08686', 'grad_norm': '19.87', 'learning_rate': '3.381e-06', 'num_tokens': '9.6e+05', 'completions/mean_length': '4.487', 'completions/min_length': '3.95', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.487', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3018', 'rewards/reward_semantic/std': '0.05605', 'rewards/reward_length/mean': '-0.04426', 'rewards/reward_length/std': '0.008602', 'reward': '1.088', 'reward_std': '0.01662', 'frac_reward_zero_std': '0.65', 'kl': '2.445', 'entropy': '0.2389', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.381', 'epoch': '0.5315'}

 33%|███▎      | 3260/10000 [3:52:54<8:07:33,  4.34s/it]
 33%|███▎      | 3261/10000 [3:52:59<8:08:07,  4.35s/it]
 33%|███▎      | 3262/10000 [3:53:03<8:02:57,  4.30s/it]
 33%|███▎      | 3263/10000 [3:53:07<8:09:10,  4.36s/it]
 33%|███▎      | 3264/10000 [3:53:12<8:07:56,  4.35s/it]
 33%|███▎      | 3265/10000 [3:53:16<8:13:13,  4.39s/it]
 33%|███▎      | 3266/10000 [3:53:21<8:14:42,  4.41s/it]
 33%|███▎      | 3267/10000 [3:53:25<8:19:54,  4.45s/it]
 33%|███▎      | 3268/10000 [3:53:29<8:11:39,  4.38s/it]
 33%|███▎      | 3269/10000 [3:53:34<8:18:26,  4.44s/it]
 33%|███▎      | 3270/10000 [3:53:39<8:22:33,  4.48s/it]
 33%|███▎      | 3271/10000 [3:53:43<8:14:01,  4.40s/it]
 33%|███▎      | 3272/10000 [3:53:47<8:14:26,  4.41s/it]
 33%|███▎      | 3273/10000 [3:53:52<8:26:52,  4.52s/it]
 33%|███▎      | 3274/10000 [3:53:56<8:15:33,  4.42s/it]
 33%|███▎      | 3275/10000 [3:54:01<8:17:12,  4.44s/it]
 33%|███▎      | 3276/10000 [3:54:05<8:01:51,  4.30s/it]
 33%|███▎      | 3277/10000 [3:54:09<8:03:26,  4.31s/it]
 33%|███▎      | 3278/10000 [3:54:14<8:13:42,  4.41s/it]
 33%|███▎      | 3279/10000 [3:54:18<8:24:58,  4.51s/it]
 33%|███▎      | 3280/10000 [3:54:23<8:14:36,  4.42s/it]
                                                        

 33%|███▎      | 3280/10000 [3:54:23<8:14:36,  4.42s/it]
 33%|███▎      | 3281/10000 [3:54:27<8:17:08,  4.44s/it]
 33%|███▎      | 3282/10000 [3:54:31<8:01:26,  4.30s/it]
 33%|███▎      | 3283/10000 [3:54:36<8:07:34,  4.36s/it]
 33%|███▎      | 3284/10000 [3:54:40<8:07:31,  4.36s/it]
 33%|███▎      | 3285/10000 [3:54:44<8:06:29,  4.35s/it]
 33%|███▎      | 3286/10000 [3:54:49<8:14:16,  4.42s/it]
 33%|███▎      | 3287/10000 [3:54:53<8:08:02,  4.36s/it]
 33%|███▎      | 3288/10000 [3:54:58<8:13:00,  4.41s/it]
 33%|███▎      | 3289/10000 [3:55:02<8:13:11,  4.41s/it]
 33%|███▎      | 3290/10000 [3:55:06<8:13:46,  4.42s/it]
 33%|███▎      | 3291/10000 [3:55:10<7:59:43,  4.29s/it]
 33%|███▎      | 3292/10000 [3:55:15<8:07:17,  4.36s/it]
 33%|███▎      | 3293/10000 [3:55:19<8:02:22,  4.32s/it]
 33%|███▎      | 3294/10000 [3:55:24<8:08:21,  4.37s/it]
 33%|███▎      | 3295/10000 [3:55:28<8:13:34,  4.42s/it]
 33%|███▎      | 3296/10000 [3:55:32<8:09:19,  4.38s/it]
 33%|███▎      | 3297/10000 [3:55:37<8:11:45,  4.40s/it]
 33%|███▎      | 3298/10000 [3:55:41<8:03:52,  4.33s/it]
 33%|███▎      | 3299/10000 [3:55:46<8:12:41,  4.41s/it]
 33%|███▎      | 3300/10000 [3:55:50<8:06:54,  4.36s/it]
                                                        

 33%|███▎      | 3300/10000 [3:55:50<8:06:54,  4.36s/it]
 33%|███▎      | 3301/10000 [3:55:54<8:14:31,  4.43s/it]
 33%|███▎      | 3302/10000 [3:55:59<8:20:04,  4.48s/it]
 33%|███▎      | 3303/10000 [3:56:04<8:22:06,  4.50s/it]
 33%|███▎      | 3304/10000 [3:56:08<8:05:21,  4.35s/it]
 33%|███▎      | 3305/10000 [3:56:12<8:01:49,  4.32s/it]
 33%|███▎      | 3306/10000 [3:56:17<8:17:28,  4.46s/it]
 33%|███▎      | 3307/10000 [3:56:21<8:12:35,  4.42s/it]
 33%|███▎      | 3308/10000 [3:56:26<8:19:16,  4.48s/it]
 33%|███▎      | 3309/10000 [3:56:30<8:21:26,  4.50s/it]
 33%|███▎      | 3310/10000 [3:56:35<8:32:04,  4.59s/it]
 33%|███▎      | 3311/10000 [3:56:39<8:20:38,  4.49s/it]
 33%|███▎      | 3312/10000 [3:56:44<8:26:17,  4.54s/it]
 33%|███▎      | 3313/10000 [3:56:48<8:07:53,  4.38s/it]
 33%|███▎      | 3314/10000 [3:56:52<8:02:53,  4.33s/it]
 33%|███▎      | 3315/10000 [3:56:57<8:08:40,  4.39s/it]
 33%|███▎      | 3316/10000 [3:57:01<8:02:58,  4.34s/it]
 33%|███▎      | 3317/10000 [3:57:05<8:10:09,  4.40s/it]
 33%|███▎      | 3318/10000 [3:57:10<8:10:58,  4.41s/it]
 33%|███▎      | 3319/10000 [3:57:15<8:26:29,  4.55s/it]
 33%|███▎      | 3320/10000 [3:57:19<8:16:01,  4.46s/it]
                                                        

 33%|███▎      | 3320/10000 [3:57:19<8:16:01,  4.46s/it]
 33%|███▎      | 3321/10000 [3:57:23<8:18:09,  4.48s/it]
 33%|███▎      | 3322/10000 [3:57:28<8:11:18,  4.41s/it]
 33%|███▎      | 3323/10000 [3:57:32<8:04:35,  4.35s/it]
 33%|███▎      | 3324/10000 [3:57:37<8:25:59,  4.55s/it]
 33%|███▎      | 3325/10000 [3:57:41<8:24:47,  4.54s/it]
 33%|███▎      | 3326/10000 [3:57:46<8:16:06,  4.46s/it]
 33%|███▎      | 3327/10000 [3:57:50<8:12:21,  4.43s/it]
 33%|███▎      | 3328/10000 [3:57:55<8:22:59,  4.52s/it]
 33%|███▎      | 3329/10000 [3:57:59<8:23:12,  4.53s/it]
 33%|███▎      | 3330/10000 [3:58:04<8:15:27,  4.46s/it]
 33%|███▎      | 3331/10000 [3:58:08<8:18:32,  4.49s/it]
 33%|███▎      | 3332/10000 [3:58:12<8:10:55,  4.42s/it]
 33%|███▎      | 3333/10000 [3:58:17<8:22:32,  4.52s/it]
 33%|███▎      | 3334/10000 [3:58:22<8:18:41,  4.49s/it]
 33%|███▎      | 3335/10000 [3:58:26<8:21:16,  4.51s/it]
 33%|███▎      | 3336/10000 [3:58:30<8:12:20,  4.43s/it]
 33%|███▎      | 3337/10000 [3:58:34<7:58:11,  4.31s/it]
 33%|███▎      | 3338/10000 [3:58:39<8:07:51,  4.39s/it]
 33%|███▎      | 3339/10000 [3:58:43<7:56:07,  4.29s/it]
 33%|███▎      | 3340/10000 [3:58:48<8:04:44,  4.37s/it]
                                                        

 33%|███▎      | 3340/10000 [3:58:48<8:04:44,  4.37s/it]
 33%|███▎      | 3341/10000 [3:58:52<7:59:50,  4.32s/it]
 33%|███▎      | 3342/10000 [3:58:56<8:07:52,  4.40s/it]
 33%|███▎      | 3343/10000 [3:59:00<7:56:18,  4.29s/it]
 33%|███▎      | 3344/10000 [3:59:05<7:54:55,  4.28s/it]
 33%|███▎      | 3345/10000 [3:59:09<8:04:07,  4.36s/it]
 33%|███▎      | 3346/10000 [3:59:14<8:01:41,  4.34s/it]
 33%|███▎      | 3347/10000 [3:59:18<8:05:50,  4.38s/it]
 33%|███▎      | 3348/10000 [3:59:22<8:01:49,  4.35s/it]
 33%|███▎      | 3349/10000 [3:59:27<8:00:58,  4.34s/it]
 34%|███▎      | 3350/10000 [3:59:31<7:56:53,  4.30s/it]
 34%|███▎      | 3351/10000 [3:59:35<7:59:45,  4.33s/it]
 34%|███▎      | 3352/10000 [3:59:40<7:58:10,  4.32s/it]
 34%|███▎      | 3353/10000 [3:59:44<7:59:55,  4.33s/it]
 34%|███▎      | 3354/10000 [3:59:48<7:52:08,  4.26s/it]
 34%|███▎      | 3355/10000 [3:59:52<7:46:18,  4.21s/it]
 34%|███▎      | 3356/10000 [3:59:56<7:50:14,  4.25s/it]
 34%|███▎      | 3357/10000 [4:00:01<7:45:36,  4.21s/it]
 34%|███▎      | 3358/10000 [4:00:05<8:02:27,  4.36s/it]
 34%|███▎      | 3359/10000 [4:00:09<7:54:01,  4.28s/it]
 34%|███▎      | 3360/10000 [4:00:14<7:58:52,  4.33s/it]
                                                        

 34%|███▎      | 3360/10000 [4:00:14<7:58:52,  4.33s/it]
 34%|███▎      | 3361/10000 [4:00:18<7:51:45,  4.26s/it]
 34%|███▎      | 3362/10000 [4:00:22<7:55:46,  4.30s/it]
 34%|███▎      | 3363/10000 [4:00:26<7:50:15,  4.25s/it]
 34%|███▎      | 3364/10000 [4:00:31<7:48:11,  4.23s/it]
 34%|███▎      | 3365/10000 [4:00:35<7:45:05,  4.21s/it]
 34%|███▎      | 3366/10000 [4:00:39<7:52:34,  4.27s/it]
 34%|███▎      | 3367/10000 [4:00:43<7:48:06,  4.23s/it]
 34%|███▎      | 3368/10000 [4:00:47<7:45:10,  4.21s/it]
 34%|███▎      | 3369/10000 [4:00:52<7:52:53,  4.28s/it]
 34%|███▎      | 3370/10000 [4:00:56<7:52:01,  4.27s/it]
 34%|███▎      | 3371/10000 [4:01:01<8:06:16,  4.40s/it]
 34%|███▎      | 3372/10000 [4:01:05<8:07:26,  4.41s/it]
 34%|███▎      | 3373/10000 [4:01:10<8:27:18,  4.59s/it]
 34%|███▎      | 3374/10000 [4:01:15<8:19:19,  4.52s/it]
 34%|███▍      | 3375/10000 [4:01:19<8:15:24,  4.49s/it]
 34%|███▍      | 3376/10000 [4:01:23<8:04:41,  4.39s/it]
 34%|███▍      | 3377/10000 [4:01:28<8:06:17,  4.41s/it]
 34%|███▍      | 3378/10000 [4:01:32<7:57:45,  4.33s/it]
 34%|███▍      | 3379/10000 [4:01:36<8:01:35,  4.36s/it]
 34%|███▍      | 3380/10000 [4:01:40<7:54:41,  4.30s/it]
                                                        

 34%|███▍      | 3380/10000 [4:01:40<7:54:41,  4.30s/it]
 34%|███▍      | 3381/10000 [4:01:45<7:56:41,  4.32s/it]
 34%|███▍      | 3382/10000 [4:01:49<7:51:22,  4.27s/it]
 34%|███▍      | 3383/10000 [4:01:54<8:05:02,  4.40s/it]
 34%|███▍      | 3384/10000 [4:01:58<7:56:44,  4.32s/it]
 34%|███▍      | 3385/10000 [4:02:02<7:51:00,  4.27s/it]
 34%|███▍      | 3386/10000 [4:02:06<7:56:30,  4.32s/it]
 34%|███▍      | 3387/10000 [4:02:11<7:50:44,  4.27s/it]
 34%|███▍      | 3388/10000 [4:02:15<8:03:19,  4.39s/it]
 34%|███▍      | 3389/10000 [4:02:20<8:04:32,  4.40s/it]
 34%|███▍      | 3390/10000 [4:02:24<8:11:31,  4.46s/it]
 34%|███▍      | 3391/10000 [4:02:28<8:01:13,  4.37s/it]
 34%|███▍      | 3392/10000 [4:02:33<8:04:01,  4.39s/it]
 34%|███▍      | 3393/10000 [4:02:37<7:56:07,  4.32s/it]
 34%|███▍      | 3394/10000 [4:02:41<8:00:06,  4.36s/it]
 34%|███▍      | 3395/10000 [4:02:46<7:53:41,  4.30s/it]
 34%|███▍      | 3396/10000 [4:02:50<7:49:08,  4.26s/it]
 34%|███▍      | 3397/10000 [4:02:54<7:50:42,  4.28s/it]
 34%|███▍      | 3398/10000 [4:02:59<8:08:36,  4.44s/it]
 34%|███▍      | 3399/10000 [4:03:03<8:09:00,  4.44s/it]
 34%|███▍      | 3400/10000 [4:03:08<7:59:24,  4.36s/it]
                                                        

 34%|███▍      | 3400/10000 [4:03:08<7:59:24,  4.36s/it]
 34%|███▍      | 3401/10000 [4:03:12<8:02:25,  4.39s/it]
 34%|███▍      | 3402/10000 [4:03:16<7:55:56,  4.33s/it]
 34%|███▍      | 3403/10000 [4:03:21<7:59:47,  4.36s/it]
 34%|███▍      | 3404/10000 [4:03:25<7:52:58,  4.30s/it]
 34%|███▍      | 3405/10000 [4:03:30<8:15:50,  4.51s/it]
 34%|███▍      | 3406/10000 [4:03:34<8:04:10,  4.41s/it]
 34%|███▍      | 3407/10000 [4:03:38<7:55:34,  4.33s/it]
 34%|███▍      | 3408/10000 [4:03:43<8:23:47,  4.59s/it]
 34%|███▍      | 3409/10000 [4:03:47<8:04:29,  4.41s/it]
 34%|███▍      | 3410/10000 [4:03:52<8:05:45,  4.42s/it]
 34%|███▍      | 3411/10000 [4:03:56<7:55:52,  4.33s/it]
 34%|███▍      | 3412/10000 [4:04:00<7:58:06,  4.35s/it]
 34%|███▍      | 3413/10000 [4:04:04<7:50:49,  4.29s/it]
 34%|███▍      | 3414/10000 [4:04:09<8:01:17,  4.38s/it]
 34%|███▍      | 3415/10000 [4:04:13<7:57:30,  4.35s/it]
 34%|███▍      | 3416/10000 [4:04:18<8:02:51,  4.40s/it]
 34%|███▍      | 3417/10000 [4:04:22<8:06:09,  4.43s/it]
 34%|███▍      | 3418/10000 [4:04:27<8:12:19,  4.49s/it]
 34%|███▍      | 3419/10000 [4:04:31<7:59:14,  4.37s/it]
 34%|███▍      | 3420/10000 [4:04:35<8:00:53,  4.38s/it]
                                                        

 34%|███▍      | 3420/10000 [4:04:35<8:00:53,  4.38s/it]
 34%|███▍      | 3421/10000 [4:04:40<7:59:17,  4.37s/it]
 34%|███▍      | 3422/10000 [4:04:45<8:12:23,  4.49s/it]
 34%|███▍      | 3423/10000 [4:04:49<8:00:41,  4.39s/it]
 34%|███▍      | 3424/10000 [4:04:53<8:04:21,  4.42s/it]
 34%|███▍      | 3425/10000 [4:04:57<7:55:05,  4.34s/it]
 34%|███▍      | 3426/10000 [4:05:02<7:58:23,  4.37s/it]
 34%|███▍      | 3427/10000 [4:05:06<7:50:34,  4.30s/it]
 34%|███▍      | 3428/10000 [4:05:10<7:54:37,  4.33s/it]
 34%|███▍      | 3429/10000 [4:05:14<7:48:01,  4.27s/it]
 34%|███▍      | 3430/10000 [4:05:19<7:54:48,  4.34s/it]
 34%|███▍      | 3431/10000 [4:05:23<7:49:00,  4.28s/it]
 34%|███▍      | 3432/10000 [4:05:28<7:54:35,  4.34s/it]
 34%|███▍      | 3433/10000 [4:05:32<7:48:31,  4.28s/it]
 34%|███▍      | 3434/10000 [4:05:36<7:53:33,  4.33s/it]
 34%|███▍      | 3435/10000 [4:05:40<7:45:06,  4.25s/it]
 34%|███▍      | 3436/10000 [4:05:45<7:51:24,  4.31s/it]
 34%|███▍      | 3437/10000 [4:05:49<7:46:09,  4.26s/it]
 34%|███▍      | 3438/10000 [4:05:53<7:55:21,  4.35s/it]
 34%|███▍      | 3439/10000 [4:05:57<7:44:58,  4.25s/it]
 34%|███▍      | 3440/10000 [4:06:02<7:54:43,  4.34s/it]
                                                        
{'loss': '0.1092', 'grad_norm': '0.02017', 'learning_rate': '3.371e-06', 'num_tokens': '9.659e+05', 'completions/mean_length': '4.731', 'completions/min_length': '4.4', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.731', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4479', 'rewards/reward_semantic/std': '0.03359', 'rewards/reward_length/mean': '-0.04023', 'rewards/reward_length/std': '0.006663', 'reward': '1.132', 'reward_std': '0.01005', 'frac_reward_zero_std': '0.65', 'kl': '2.628', 'entropy': '0.1747', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.368', 'epoch': '0.5348'}
{'loss': '0.1092', 'grad_norm': '0.01658', 'learning_rate': '3.361e-06', 'num_tokens': '9.716e+05', 'completions/mean_length': '4.6', 'completions/min_length': '4.05', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.6', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3694', 'rewards/reward_semantic/std': '0.0965', 'rewards/reward_length/mean': '-0.04429', 'rewards/reward_length/std': '0.01095', 'reward': '1.109', 'reward_std': '0.02884', 'frac_reward_zero_std': '0.4', 'kl': '2.527', 'entropy': '0.2834', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.391', 'epoch': '0.5381'}
{'loss': '0.07979', 'grad_norm': '0.03965', 'learning_rate': '3.351e-06', 'num_tokens': '9.775e+05', 'completions/mean_length': '4.381', 'completions/min_length': '3.85', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.381', 'completions/min_terminated_length': '3.85', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3277', 'rewards/reward_semantic/std': '0.03098', 'rewards/reward_length/mean': '-0.03667', 'rewards/reward_length/std': '0.009324', 'reward': '1.096', 'reward_std': '0.009434', 'frac_reward_zero_std': '0.6', 'kl': '2.669', 'entropy': '0.1727', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.346', 'epoch': '0.5413'}
{'loss': '0.1022', 'grad_norm': '0.09227', 'learning_rate': '3.341e-06', 'num_tokens': '9.835e+05', 'completions/mean_length': '4.888', 'completions/min_length': '4.6', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.888', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5053', 'rewards/reward_semantic/std': '0.04352', 'rewards/reward_length/mean': '-0.03594', 'rewards/reward_length/std': '0.008613', 'reward': '1.15', 'reward_std': '0.01319', 'frac_reward_zero_std': '0.55', 'kl': '2.735', 'entropy': '0.1382', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.431', 'epoch': '0.5446'}
{'loss': '0.0681', 'grad_norm': '0.1456', 'learning_rate': '3.331e-06', 'num_tokens': '9.894e+05', 'completions/mean_length': '4.831', 'completions/min_length': '4.3', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.831', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3009', 'rewards/reward_semantic/std': '0.04895', 'rewards/reward_length/mean': '-0.03245', 'rewards/reward_length/std': '0.0107', 'reward': '1.089', 'reward_std': '0.01432', 'frac_reward_zero_std': '0.65', 'kl': '2.82', 'entropy': '0.1519', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.416', 'epoch': '0.5479'}
{'loss': '0.1699', 'grad_norm': '0.0003109', 'learning_rate': '3.321e-06', 'num_tokens': '9.951e+05', 'completions/mean_length': '4.4', 'completions/min_length': '4.2', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.4', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3705', 'rewards/reward_semantic/std': '0.06608', 'rewards/reward_length/mean': '-0.02539', 'rewards/reward_length/std': '0.005021', 'reward': '1.11', 'reward_std': '0.01986', 'frac_reward_zero_std': '0.8', 'kl': '3.312', 'entropy': '0.07528', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.288', 'epoch': '0.5512'}
{'loss': '0.09741', 'grad_norm': '0.009913', 'learning_rate': '3.311e-06', 'num_tokens': '1.001e+06', 'completions/mean_length': '4.956', 'completions/min_length': '4.6', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.956', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2624', 'rewards/reward_semantic/std': '0.02186', 'rewards/reward_length/mean': '-0.03076', 'rewards/reward_length/std': '0.007264', 'reward': '1.077', 'reward_std': '0.006509', 'frac_reward_zero_std': '0.75', 'kl': '3.05', 'entropy': '0.1147', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.315', 'epoch': '0.5545'}
{'loss': '0.1393', 'grad_norm': '0.04993', 'learning_rate': '3.301e-06', 'num_tokens': '1.007e+06', 'completions/mean_length': '4.912', 'completions/min_length': '4.7', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.912', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1571', 'rewards/reward_semantic/std': '0.0388', 'rewards/reward_length/mean': '-0.02922', 'rewards/reward_length/std': '0.005378', 'reward': '1.046', 'reward_std': '0.01174', 'frac_reward_zero_std': '0.75', 'kl': '2.529', 'entropy': '0.09529', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.336', 'epoch': '0.5577'}
{'loss': '0.1502', 'grad_norm': '0.0006053', 'learning_rate': '3.291e-06', 'num_tokens': '1.013e+06', 'completions/mean_length': '5.056', 'completions/min_length': '4.45', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.056', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1941', 'rewards/reward_semantic/std': '0.08405', 'rewards/reward_length/mean': '-0.03433', 'rewards/reward_length/std': '0.01234', 'reward': '1.057', 'reward_std': '0.0252', 'frac_reward_zero_std': '0.55', 'kl': '2.836', 'entropy': '0.1417', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.376', 'epoch': '0.561'}

 34%|███▍      | 3440/10000 [4:06:02<7:54:43,  4.34s/it]
 34%|███▍      | 3441/10000 [4:06:06<7:52:13,  4.32s/it]
 34%|███▍      | 3442/10000 [4:06:11<8:03:49,  4.43s/it]
 34%|███▍      | 3443/10000 [4:06:15<7:50:23,  4.30s/it]
 34%|███▍      | 3444/10000 [4:06:20<8:05:19,  4.44s/it]
 34%|███▍      | 3445/10000 [4:06:24<7:59:33,  4.39s/it]
 34%|███▍      | 3446/10000 [4:06:28<8:04:48,  4.44s/it]
 34%|███▍      | 3447/10000 [4:06:33<7:58:52,  4.38s/it]
 34%|███▍      | 3448/10000 [4:06:37<8:04:18,  4.44s/it]
 34%|███▍      | 3449/10000 [4:06:42<7:59:10,  4.39s/it]
 34%|███▍      | 3450/10000 [4:06:46<8:05:31,  4.45s/it]
 35%|███▍      | 3451/10000 [4:06:50<7:59:54,  4.40s/it]
 35%|███▍      | 3452/10000 [4:06:55<8:04:00,  4.43s/it]
 35%|███▍      | 3453/10000 [4:06:59<7:58:06,  4.38s/it]
 35%|███▍      | 3454/10000 [4:07:04<8:04:12,  4.44s/it]
 35%|███▍      | 3455/10000 [4:07:08<7:58:44,  4.39s/it]
 35%|███▍      | 3456/10000 [4:07:13<8:05:02,  4.45s/it]
 35%|███▍      | 3457/10000 [4:07:17<7:51:21,  4.32s/it]
 35%|███▍      | 3458/10000 [4:07:21<7:59:58,  4.40s/it]
 35%|███▍      | 3459/10000 [4:07:25<7:53:53,  4.35s/it]
 35%|███▍      | 3460/10000 [4:07:30<7:59:36,  4.40s/it]
                                                        

 35%|███▍      | 3460/10000 [4:07:30<7:59:36,  4.40s/it]
 35%|███▍      | 3461/10000 [4:07:34<7:55:13,  4.36s/it]
 35%|███▍      | 3462/10000 [4:07:39<8:05:15,  4.45s/it]
 35%|███▍      | 3463/10000 [4:07:44<8:09:58,  4.50s/it]
 35%|███▍      | 3464/10000 [4:07:48<8:11:23,  4.51s/it]
 35%|███▍      | 3465/10000 [4:07:52<8:02:27,  4.43s/it]
 35%|███▍      | 3466/10000 [4:07:57<8:05:40,  4.46s/it]
 35%|███▍      | 3467/10000 [4:08:01<7:58:03,  4.39s/it]
 35%|███▍      | 3468/10000 [4:08:06<8:05:30,  4.46s/it]
 35%|███▍      | 3469/10000 [4:08:10<8:06:13,  4.47s/it]
 35%|███▍      | 3470/10000 [4:08:15<8:02:28,  4.43s/it]
 35%|███▍      | 3471/10000 [4:08:19<7:56:32,  4.38s/it]
 35%|███▍      | 3472/10000 [4:08:23<8:00:50,  4.42s/it]
 35%|███▍      | 3473/10000 [4:08:28<7:58:12,  4.40s/it]
 35%|███▍      | 3474/10000 [4:08:32<8:02:50,  4.44s/it]
 35%|███▍      | 3475/10000 [4:08:36<7:48:55,  4.31s/it]
 35%|███▍      | 3476/10000 [4:08:41<8:00:04,  4.42s/it]
 35%|███▍      | 3477/10000 [4:08:45<7:47:12,  4.30s/it]
 35%|███▍      | 3478/10000 [4:08:49<7:54:43,  4.37s/it]
 35%|███▍      | 3479/10000 [4:08:54<7:50:46,  4.33s/it]
 35%|███▍      | 3480/10000 [4:08:58<7:56:35,  4.39s/it]
                                                        

 35%|███▍      | 3480/10000 [4:08:58<7:56:35,  4.39s/it]
 35%|███▍      | 3481/10000 [4:09:02<7:52:01,  4.34s/it]
 35%|███▍      | 3482/10000 [4:09:07<8:10:23,  4.51s/it]
 35%|███▍      | 3483/10000 [4:09:12<8:13:05,  4.54s/it]
 35%|███▍      | 3484/10000 [4:09:16<8:12:59,  4.54s/it]
 35%|███▍      | 3485/10000 [4:09:21<8:09:55,  4.51s/it]
 35%|███▍      | 3486/10000 [4:09:25<8:10:41,  4.52s/it]
 35%|███▍      | 3487/10000 [4:09:30<8:00:24,  4.43s/it]
 35%|███▍      | 3488/10000 [4:09:34<8:01:23,  4.44s/it]
 35%|███▍      | 3489/10000 [4:09:38<7:54:56,  4.38s/it]
 35%|███▍      | 3490/10000 [4:09:43<7:52:37,  4.36s/it]
 35%|███▍      | 3491/10000 [4:09:47<7:48:38,  4.32s/it]
 35%|███▍      | 3492/10000 [4:09:51<7:55:54,  4.39s/it]
 35%|███▍      | 3493/10000 [4:09:56<7:51:52,  4.35s/it]
 35%|███▍      | 3494/10000 [4:10:00<7:59:59,  4.43s/it]
 35%|███▍      | 3495/10000 [4:10:04<7:46:37,  4.30s/it]
 35%|███▍      | 3496/10000 [4:10:09<7:51:38,  4.35s/it]
 35%|███▍      | 3497/10000 [4:10:13<7:48:05,  4.32s/it]
 35%|███▍      | 3498/10000 [4:10:18<7:55:14,  4.39s/it]
 35%|███▍      | 3499/10000 [4:10:22<7:50:31,  4.34s/it]
 35%|███▌      | 3500/10000 [4:10:26<7:56:20,  4.40s/it]
                                                        

 35%|███▌      | 3500/10000 [4:10:26<7:56:20,  4.40s/it]
 35%|███▌      | 3501/10000 [4:10:32<8:25:38,  4.67s/it]
 35%|███▌      | 3502/10000 [4:10:36<8:21:14,  4.63s/it]
 35%|███▌      | 3503/10000 [4:10:40<8:08:50,  4.51s/it]
 35%|███▌      | 3504/10000 [4:10:45<8:09:42,  4.52s/it]
 35%|███▌      | 3505/10000 [4:10:49<8:04:39,  4.48s/it]
 35%|███▌      | 3506/10000 [4:10:54<8:05:15,  4.48s/it]
 35%|███▌      | 3507/10000 [4:10:58<7:55:36,  4.39s/it]
 35%|███▌      | 3508/10000 [4:11:03<8:01:19,  4.45s/it]
 35%|███▌      | 3509/10000 [4:11:07<7:54:07,  4.38s/it]
 35%|███▌      | 3510/10000 [4:11:11<7:57:52,  4.42s/it]
 35%|███▌      | 3511/10000 [4:11:16<7:51:52,  4.36s/it]
 35%|███▌      | 3512/10000 [4:11:20<8:00:13,  4.44s/it]
 35%|███▌      | 3513/10000 [4:11:24<7:53:34,  4.38s/it]
 35%|███▌      | 3514/10000 [4:11:29<7:58:25,  4.43s/it]
 35%|███▌      | 3515/10000 [4:11:33<7:53:05,  4.38s/it]
 35%|███▌      | 3516/10000 [4:11:38<7:57:36,  4.42s/it]
 35%|███▌      | 3517/10000 [4:11:42<7:50:38,  4.36s/it]
 35%|███▌      | 3518/10000 [4:11:46<7:55:42,  4.40s/it]
 35%|███▌      | 3519/10000 [4:11:51<7:49:24,  4.35s/it]
 35%|███▌      | 3520/10000 [4:11:55<7:46:48,  4.32s/it]
                                                        

 35%|███▌      | 3520/10000 [4:11:55<7:46:48,  4.32s/it]
 35%|███▌      | 3521/10000 [4:11:59<7:42:02,  4.28s/it]
 35%|███▌      | 3522/10000 [4:12:04<7:49:42,  4.35s/it]
 35%|███▌      | 3523/10000 [4:12:08<7:45:56,  4.32s/it]
 35%|███▌      | 3524/10000 [4:12:12<7:52:48,  4.38s/it]
 35%|███▌      | 3525/10000 [4:12:17<7:47:47,  4.33s/it]
 35%|███▌      | 3526/10000 [4:12:21<7:52:01,  4.37s/it]
 35%|███▌      | 3527/10000 [4:12:25<7:47:06,  4.33s/it]
 35%|███▌      | 3528/10000 [4:12:30<7:59:11,  4.44s/it]
 35%|███▌      | 3529/10000 [4:12:34<7:51:51,  4.38s/it]
 35%|███▌      | 3530/10000 [4:12:39<7:56:01,  4.41s/it]
 35%|███▌      | 3531/10000 [4:12:43<7:49:59,  4.36s/it]
 35%|███▌      | 3532/10000 [4:12:47<7:55:10,  4.41s/it]
 35%|███▌      | 3533/10000 [4:12:52<7:49:01,  4.35s/it]
 35%|███▌      | 3534/10000 [4:12:56<7:54:04,  4.40s/it]
 35%|███▌      | 3535/10000 [4:13:01<8:18:56,  4.63s/it]
 35%|███▌      | 3536/10000 [4:13:06<8:05:34,  4.51s/it]
 35%|███▌      | 3537/10000 [4:13:10<8:05:16,  4.51s/it]
 35%|███▌      | 3538/10000 [4:13:14<7:56:04,  4.42s/it]
 35%|███▌      | 3539/10000 [4:13:19<8:10:42,  4.56s/it]
 35%|███▌      | 3540/10000 [4:13:23<7:59:37,  4.45s/it]
                                                        

 35%|███▌      | 3540/10000 [4:13:23<7:59:37,  4.45s/it]
 35%|███▌      | 3541/10000 [4:13:28<8:03:11,  4.49s/it]
 35%|███▌      | 3542/10000 [4:13:32<7:47:01,  4.34s/it]
 35%|███▌      | 3543/10000 [4:13:36<7:50:13,  4.37s/it]
 35%|███▌      | 3544/10000 [4:13:41<7:41:42,  4.29s/it]
 35%|███▌      | 3545/10000 [4:13:45<7:42:26,  4.30s/it]
 35%|███▌      | 3546/10000 [4:13:49<7:35:36,  4.24s/it]
 35%|███▌      | 3547/10000 [4:13:53<7:44:05,  4.32s/it]
 35%|███▌      | 3548/10000 [4:13:58<7:39:06,  4.27s/it]
 35%|███▌      | 3549/10000 [4:14:02<7:45:50,  4.33s/it]
 36%|███▌      | 3550/10000 [4:14:06<7:40:51,  4.29s/it]
 36%|███▌      | 3551/10000 [4:14:11<7:50:32,  4.38s/it]
 36%|███▌      | 3552/10000 [4:14:15<7:44:16,  4.32s/it]
 36%|███▌      | 3553/10000 [4:14:20<7:53:14,  4.40s/it]
 36%|███▌      | 3554/10000 [4:14:24<7:43:57,  4.32s/it]
 36%|███▌      | 3555/10000 [4:14:28<7:47:16,  4.35s/it]
 36%|███▌      | 3556/10000 [4:14:32<7:33:02,  4.22s/it]
 36%|███▌      | 3557/10000 [4:14:37<7:41:36,  4.30s/it]
 36%|███▌      | 3558/10000 [4:14:41<7:36:26,  4.25s/it]
 36%|███▌      | 3559/10000 [4:14:45<7:34:54,  4.24s/it]
 36%|███▌      | 3560/10000 [4:14:49<7:32:04,  4.21s/it]
                                                        

 36%|███▌      | 3560/10000 [4:14:49<7:32:04,  4.21s/it]
 36%|███▌      | 3561/10000 [4:14:54<7:39:18,  4.28s/it]
 36%|███▌      | 3562/10000 [4:14:58<7:34:06,  4.23s/it]
 36%|███▌      | 3563/10000 [4:15:03<7:55:09,  4.43s/it]
 36%|███▌      | 3564/10000 [4:15:07<7:45:35,  4.34s/it]
 36%|███▌      | 3565/10000 [4:15:11<7:48:19,  4.37s/it]
 36%|███▌      | 3566/10000 [4:15:15<7:41:09,  4.30s/it]
 36%|███▌      | 3567/10000 [4:15:20<7:45:29,  4.34s/it]
 36%|███▌      | 3568/10000 [4:15:24<7:36:12,  4.26s/it]
 36%|███▌      | 3569/10000 [4:15:28<7:41:49,  4.31s/it]
 36%|███▌      | 3570/10000 [4:15:32<7:35:24,  4.25s/it]
 36%|███▌      | 3571/10000 [4:15:37<7:39:49,  4.29s/it]
 36%|███▌      | 3572/10000 [4:15:41<7:30:57,  4.21s/it]
 36%|███▌      | 3573/10000 [4:15:45<7:36:53,  4.27s/it]
 36%|███▌      | 3574/10000 [4:15:49<7:32:08,  4.22s/it]
 36%|███▌      | 3575/10000 [4:15:54<7:48:36,  4.38s/it]
 36%|███▌      | 3576/10000 [4:15:58<7:39:57,  4.30s/it]
 36%|███▌      | 3577/10000 [4:16:02<7:44:01,  4.33s/it]
 36%|███▌      | 3578/10000 [4:16:07<7:41:37,  4.31s/it]
 36%|███▌      | 3579/10000 [4:16:11<7:45:14,  4.35s/it]
 36%|███▌      | 3580/10000 [4:16:15<7:40:53,  4.31s/it]
                                                        

 36%|███▌      | 3580/10000 [4:16:15<7:40:53,  4.31s/it]
 36%|███▌      | 3581/10000 [4:16:20<7:44:22,  4.34s/it]
 36%|███▌      | 3582/10000 [4:16:24<7:37:49,  4.28s/it]
 36%|███▌      | 3583/10000 [4:16:28<7:43:53,  4.34s/it]
 36%|███▌      | 3584/10000 [4:16:33<7:37:53,  4.28s/it]
 36%|███▌      | 3585/10000 [4:16:37<7:34:09,  4.25s/it]
 36%|███▌      | 3586/10000 [4:16:41<7:29:32,  4.21s/it]
 36%|███▌      | 3587/10000 [4:16:46<7:45:04,  4.35s/it]
 36%|███▌      | 3588/10000 [4:16:50<7:42:58,  4.33s/it]
 36%|███▌      | 3589/10000 [4:16:55<8:02:26,  4.52s/it]
 36%|███▌      | 3590/10000 [4:16:59<7:54:51,  4.44s/it]
 36%|███▌      | 3591/10000 [4:17:04<7:57:56,  4.47s/it]
 36%|███▌      | 3592/10000 [4:17:08<7:51:26,  4.41s/it]
 36%|███▌      | 3593/10000 [4:17:12<7:56:33,  4.46s/it]
 36%|███▌      | 3594/10000 [4:17:17<7:51:07,  4.41s/it]
 36%|███▌      | 3595/10000 [4:17:21<7:48:25,  4.39s/it]
 36%|███▌      | 3596/10000 [4:17:25<7:45:00,  4.36s/it]
 36%|███▌      | 3597/10000 [4:17:30<7:51:49,  4.42s/it]
 36%|███▌      | 3598/10000 [4:17:34<7:47:49,  4.38s/it]
 36%|███▌      | 3599/10000 [4:17:39<7:53:19,  4.44s/it]
 36%|███▌      | 3600/10000 [4:17:43<7:47:11,  4.38s/it]
                                                        

 36%|███▌      | 3600/10000 [4:17:43<7:47:11,  4.38s/it]
 36%|███▌      | 3601/10000 [4:17:48<7:52:03,  4.43s/it]
 36%|███▌      | 3602/10000 [4:17:52<7:37:44,  4.29s/it]
 36%|███▌      | 3603/10000 [4:17:56<7:37:12,  4.29s/it]
 36%|███▌      | 3604/10000 [4:18:00<7:36:01,  4.28s/it]
 36%|███▌      | 3605/10000 [4:18:05<7:47:49,  4.39s/it]
 36%|███▌      | 3606/10000 [4:18:09<7:43:28,  4.35s/it]
 36%|███▌      | 3607/10000 [4:18:13<7:49:00,  4.40s/it]
 36%|███▌      | 3608/10000 [4:18:18<7:44:01,  4.36s/it]
 36%|███▌      | 3609/10000 [4:18:22<7:50:07,  4.41s/it]
 36%|███▌      | 3610/10000 [4:18:27<8:02:18,  4.53s/it]
 36%|███▌      | 3611/10000 [4:18:32<8:16:46,  4.67s/it]
 36%|███▌      | 3612/10000 [4:18:36<7:56:04,  4.47s/it]
 36%|███▌      | 3613/10000 [4:18:41<7:58:06,  4.49s/it]
 36%|███▌      | 3614/10000 [4:18:45<7:50:21,  4.42s/it]
 36%|███▌      | 3615/10000 [4:18:49<7:53:16,  4.45s/it]
 36%|███▌      | 3616/10000 [4:18:54<7:46:03,  4.38s/it]
 36%|███▌      | 3617/10000 [4:18:58<7:48:19,  4.40s/it]
 36%|███▌      | 3618/10000 [4:19:02<7:42:08,  4.34s/it]
 36%|███▌      | 3619/10000 [4:19:07<7:47:39,  4.40s/it]
 36%|███▌      | 3620/10000 [4:19:11<7:42:27,  4.35s/it]
                                                        
{'loss': '0.1132', 'grad_norm': '0.3206', 'learning_rate': '3.281e-06', 'num_tokens': '1.019e+06', 'completions/mean_length': '4.819', 'completions/min_length': '4.55', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.819', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3652', 'rewards/reward_semantic/std': '0.05515', 'rewards/reward_length/mean': '-0.03589', 'rewards/reward_length/std': '0.005833', 'reward': '1.108', 'reward_std': '0.01646', 'frac_reward_zero_std': '0.75', 'kl': '2.76', 'entropy': '0.1208', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.306', 'epoch': '0.5643'}
{'loss': '0.106', 'grad_norm': '0.002999', 'learning_rate': '3.271e-06', 'num_tokens': '1.025e+06', 'completions/mean_length': '4.325', 'completions/min_length': '4.05', 'completions/max_length': '4.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.325', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3823', 'rewards/reward_semantic/std': '0.002946', 'rewards/reward_length/mean': '-0.02527', 'rewards/reward_length/std': '0.003423', 'reward': '1.113', 'reward_std': '0.001055', 'frac_reward_zero_std': '0.8', 'kl': '3.022', 'entropy': '0.08218', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.383', 'epoch': '0.5676'}
{'loss': '0.1012', 'grad_norm': '0.01847', 'learning_rate': '3.261e-06', 'num_tokens': '1.031e+06', 'completions/mean_length': '4.456', 'completions/min_length': '4.05', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.456', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3432', 'rewards/reward_semantic/std': '0.05295', 'rewards/reward_length/mean': '-0.03562', 'rewards/reward_length/std': '0.008145', 'reward': '1.101', 'reward_std': '0.01578', 'frac_reward_zero_std': '0.6', 'kl': '2.926', 'entropy': '0.1242', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.389', 'epoch': '0.5709'}
{'loss': '0.1083', 'grad_norm': '21.1', 'learning_rate': '3.251e-06', 'num_tokens': '1.037e+06', 'completions/mean_length': '4.388', 'completions/min_length': '3.95', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.388', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3174', 'rewards/reward_semantic/std': '0.06702', 'rewards/reward_length/mean': '-0.02852', 'rewards/reward_length/std': '0.008429', 'reward': '1.094', 'reward_std': '0.01987', 'frac_reward_zero_std': '0.7', 'kl': '3.059', 'entropy': '0.1138', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.389', 'epoch': '0.5741'}
{'loss': '0.1244', 'grad_norm': '0.03121', 'learning_rate': '3.241e-06', 'num_tokens': '1.043e+06', 'completions/mean_length': '4.656', 'completions/min_length': '4.45', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.656', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2562', 'rewards/reward_semantic/std': '0.03493', 'rewards/reward_length/mean': '-0.02466', 'rewards/reward_length/std': '0.004144', 'reward': '1.076', 'reward_std': '0.0106', 'frac_reward_zero_std': '0.8', 'kl': '2.943', 'entropy': '0.108', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.357', 'epoch': '0.5774'}
{'loss': '0.09386', 'grad_norm': '0.007344', 'learning_rate': '3.231e-06', 'num_tokens': '1.049e+06', 'completions/mean_length': '5.1', 'completions/min_length': '4.75', 'completions/max_length': '5.55', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.1', 'completions/min_terminated_length': '4.75', 'completions/max_terminated_length': '5.55', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1786', 'rewards/reward_semantic/std': '0.04253', 'rewards/reward_length/mean': '-0.02395', 'rewards/reward_length/std': '0.008861', 'reward': '1.052', 'reward_std': '0.01274', 'frac_reward_zero_std': '0.7', 'kl': '2.969', 'entropy': '0.1074', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.405', 'epoch': '0.5807'}
{'loss': '0.1113', 'grad_norm': '0.0215', 'learning_rate': '3.221e-06', 'num_tokens': '1.055e+06', 'completions/mean_length': '4.156', 'completions/min_length': '3.9', 'completions/max_length': '4.4', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.156', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '4.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3406', 'rewards/reward_semantic/std': '0.0186', 'rewards/reward_length/mean': '-0.03218', 'rewards/reward_length/std': '0.005246', 'reward': '1.101', 'reward_std': '0.005824', 'frac_reward_zero_std': '0.7', 'kl': '2.974', 'entropy': '0.1396', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.263', 'epoch': '0.584'}
{'loss': '0.09527', 'grad_norm': '0.01113', 'learning_rate': '3.211e-06', 'num_tokens': '1.061e+06', 'completions/mean_length': '5.05', 'completions/min_length': '4.75', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.05', 'completions/min_terminated_length': '4.75', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3146', 'rewards/reward_semantic/std': '0.03946', 'rewards/reward_length/mean': '-0.02854', 'rewards/reward_length/std': '0.00253', 'reward': '1.093', 'reward_std': '0.01184', 'frac_reward_zero_std': '0.75', 'kl': '2.611', 'entropy': '0.07991', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.296', 'epoch': '0.5873'}
{'loss': '0.1215', 'grad_norm': '0.01821', 'learning_rate': '3.201e-06', 'num_tokens': '1.067e+06', 'completions/mean_length': '4.662', 'completions/min_length': '4.3', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.662', 'completions/min_terminated_length': '4.3', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2563', 'rewards/reward_semantic/std': '0.03586', 'rewards/reward_length/mean': '-0.03064', 'rewards/reward_length/std': '0.005974', 'reward': '1.075', 'reward_std': '0.01073', 'frac_reward_zero_std': '0.65', 'kl': '3.082', 'entropy': '0.1056', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.362', 'epoch': '0.5906'}

 36%|███▌      | 3620/10000 [4:19:11<7:42:27,  4.35s/it]
 36%|███▌      | 3621/10000 [4:19:16<7:47:39,  4.40s/it]
 36%|███▌      | 3622/10000 [4:19:20<7:41:41,  4.34s/it]
 36%|███▌      | 3623/10000 [4:19:24<7:46:40,  4.39s/it]
 36%|███▌      | 3624/10000 [4:19:29<7:47:37,  4.40s/it]
 36%|███▋      | 3625/10000 [4:19:33<7:51:21,  4.44s/it]
 36%|███▋      | 3626/10000 [4:19:37<7:44:25,  4.37s/it]
 36%|███▋      | 3627/10000 [4:19:42<7:49:05,  4.42s/it]
 36%|███▋      | 3628/10000 [4:19:46<7:41:48,  4.35s/it]
 36%|███▋      | 3629/10000 [4:19:51<7:47:01,  4.40s/it]
 36%|███▋      | 3630/10000 [4:19:55<7:34:33,  4.28s/it]
 36%|███▋      | 3631/10000 [4:19:59<7:44:18,  4.37s/it]
 36%|███▋      | 3632/10000 [4:20:03<7:39:56,  4.33s/it]
 36%|███▋      | 3633/10000 [4:20:08<7:45:32,  4.39s/it]
 36%|███▋      | 3634/10000 [4:20:12<7:41:22,  4.35s/it]
 36%|███▋      | 3635/10000 [4:20:17<7:49:57,  4.43s/it]
 36%|███▋      | 3636/10000 [4:20:21<7:52:27,  4.45s/it]
 36%|███▋      | 3637/10000 [4:20:26<7:44:36,  4.38s/it]
 36%|███▋      | 3638/10000 [4:20:30<7:48:33,  4.42s/it]
 36%|███▋      | 3639/10000 [4:20:34<7:41:58,  4.36s/it]
 36%|███▋      | 3640/10000 [4:20:39<7:46:32,  4.40s/it]
                                                        

 36%|███▋      | 3640/10000 [4:20:39<7:46:32,  4.40s/it]
 36%|███▋      | 3641/10000 [4:20:43<7:54:39,  4.48s/it]
 36%|███▋      | 3642/10000 [4:20:48<7:54:28,  4.48s/it]
 36%|███▋      | 3643/10000 [4:20:52<7:46:42,  4.40s/it]
 36%|███▋      | 3644/10000 [4:20:57<7:50:22,  4.44s/it]
 36%|███▋      | 3645/10000 [4:21:01<7:42:32,  4.37s/it]
 36%|███▋      | 3646/10000 [4:21:05<7:47:34,  4.42s/it]
 36%|███▋      | 3647/10000 [4:21:10<7:44:55,  4.39s/it]
 36%|███▋      | 3648/10000 [4:21:14<7:46:16,  4.40s/it]
 36%|███▋      | 3649/10000 [4:21:19<7:43:41,  4.38s/it]
 36%|███▋      | 3650/10000 [4:21:23<7:45:17,  4.40s/it]
 37%|███▋      | 3651/10000 [4:21:27<7:38:00,  4.33s/it]
 37%|███▋      | 3652/10000 [4:21:32<7:42:19,  4.37s/it]
 37%|███▋      | 3653/10000 [4:21:36<7:35:24,  4.31s/it]
 37%|███▋      | 3654/10000 [4:21:40<7:40:52,  4.36s/it]
 37%|███▋      | 3655/10000 [4:21:44<7:34:29,  4.30s/it]
 37%|███▋      | 3656/10000 [4:21:49<7:39:52,  4.35s/it]
 37%|███▋      | 3657/10000 [4:21:53<7:38:41,  4.34s/it]
 37%|███▋      | 3658/10000 [4:21:58<7:42:54,  4.38s/it]
 37%|███▋      | 3659/10000 [4:22:02<7:35:54,  4.31s/it]
 37%|███▋      | 3660/10000 [4:22:06<7:40:16,  4.36s/it]
                                                        

 37%|███▋      | 3660/10000 [4:22:06<7:40:16,  4.36s/it]
 37%|███▋      | 3661/10000 [4:22:10<7:26:40,  4.23s/it]
 37%|███▋      | 3662/10000 [4:22:15<7:33:37,  4.29s/it]
 37%|███▋      | 3663/10000 [4:22:19<7:28:46,  4.25s/it]
 37%|███▋      | 3664/10000 [4:22:24<7:46:43,  4.42s/it]
 37%|███▋      | 3665/10000 [4:22:28<7:38:59,  4.35s/it]
 37%|███▋      | 3666/10000 [4:22:32<7:34:39,  4.31s/it]
 37%|███▋      | 3667/10000 [4:22:36<7:32:34,  4.29s/it]
 37%|███▋      | 3668/10000 [4:22:41<7:38:24,  4.34s/it]
 37%|███▋      | 3669/10000 [4:22:45<7:31:59,  4.28s/it]
 37%|███▋      | 3670/10000 [4:22:50<7:43:31,  4.39s/it]
 37%|███▋      | 3671/10000 [4:22:54<7:35:30,  4.32s/it]
 37%|███▋      | 3672/10000 [4:22:58<7:41:47,  4.38s/it]
 37%|███▋      | 3673/10000 [4:23:02<7:38:38,  4.35s/it]
 37%|███▋      | 3674/10000 [4:23:07<7:37:19,  4.34s/it]
 37%|███▋      | 3675/10000 [4:23:11<7:34:35,  4.31s/it]
 37%|███▋      | 3676/10000 [4:23:16<7:45:53,  4.42s/it]
 37%|███▋      | 3677/10000 [4:23:20<7:33:23,  4.30s/it]
 37%|███▋      | 3678/10000 [4:23:24<7:40:57,  4.37s/it]
 37%|███▋      | 3679/10000 [4:23:29<7:36:51,  4.34s/it]
 37%|███▋      | 3680/10000 [4:23:33<7:43:48,  4.40s/it]
                                                        

 37%|███▋      | 3680/10000 [4:23:33<7:43:48,  4.40s/it]
 37%|███▋      | 3681/10000 [4:23:37<7:39:04,  4.36s/it]
 37%|███▋      | 3682/10000 [4:23:43<8:15:16,  4.70s/it]
 37%|███▋      | 3683/10000 [4:23:47<8:01:22,  4.57s/it]
 37%|███▋      | 3684/10000 [4:23:52<8:04:53,  4.61s/it]
 37%|███▋      | 3685/10000 [4:23:56<7:54:26,  4.51s/it]
 37%|███▋      | 3686/10000 [4:24:01<7:55:50,  4.52s/it]
 37%|███▋      | 3687/10000 [4:24:05<7:48:19,  4.45s/it]
 37%|███▋      | 3688/10000 [4:24:10<7:55:47,  4.52s/it]
 37%|███▋      | 3689/10000 [4:24:14<7:47:41,  4.45s/it]
 37%|███▋      | 3690/10000 [4:24:18<7:51:33,  4.48s/it]
 37%|███▋      | 3691/10000 [4:24:23<7:44:19,  4.42s/it]
 37%|███▋      | 3692/10000 [4:24:27<7:48:53,  4.46s/it]
 37%|███▋      | 3693/10000 [4:24:32<7:43:07,  4.41s/it]
 37%|███▋      | 3694/10000 [4:24:36<7:48:09,  4.45s/it]
 37%|███▋      | 3695/10000 [4:24:40<7:42:19,  4.40s/it]
 37%|███▋      | 3696/10000 [4:24:45<7:40:23,  4.38s/it]
 37%|███▋      | 3697/10000 [4:24:49<7:37:19,  4.35s/it]
 37%|███▋      | 3698/10000 [4:24:53<7:41:53,  4.40s/it]
 37%|███▋      | 3699/10000 [4:24:58<7:36:54,  4.35s/it]
 37%|███▋      | 3700/10000 [4:25:02<7:40:15,  4.38s/it]
                                                        

 37%|███▋      | 3700/10000 [4:25:02<7:40:15,  4.38s/it]
 37%|███▋      | 3701/10000 [4:25:06<7:36:21,  4.35s/it]
 37%|███▋      | 3702/10000 [4:25:11<7:42:28,  4.41s/it]
 37%|███▋      | 3703/10000 [4:25:15<7:37:31,  4.36s/it]
 37%|███▋      | 3704/10000 [4:25:20<7:42:51,  4.41s/it]
 37%|███▋      | 3705/10000 [4:25:25<8:05:15,  4.63s/it]
 37%|███▋      | 3706/10000 [4:25:29<7:52:43,  4.51s/it]
 37%|███▋      | 3707/10000 [4:25:34<7:54:25,  4.52s/it]
 37%|███▋      | 3708/10000 [4:25:38<7:45:57,  4.44s/it]
 37%|███▋      | 3709/10000 [4:25:43<7:49:25,  4.48s/it]
 37%|███▋      | 3710/10000 [4:25:47<7:42:36,  4.41s/it]
 37%|███▋      | 3711/10000 [4:25:52<8:08:51,  4.66s/it]
 37%|███▋      | 3712/10000 [4:25:56<7:56:08,  4.54s/it]
 37%|███▋      | 3713/10000 [4:26:01<8:09:06,  4.67s/it]
 37%|███▋      | 3714/10000 [4:26:06<7:56:25,  4.55s/it]
 37%|███▋      | 3715/10000 [4:26:10<7:56:46,  4.55s/it]
 37%|███▋      | 3716/10000 [4:26:15<8:03:38,  4.62s/it]
 37%|███▋      | 3717/10000 [4:26:19<8:02:07,  4.60s/it]
 37%|███▋      | 3718/10000 [4:26:24<7:51:20,  4.50s/it]
 37%|███▋      | 3719/10000 [4:26:28<7:53:09,  4.52s/it]
 37%|███▋      | 3720/10000 [4:26:33<7:45:17,  4.45s/it]
                                                        

 37%|███▋      | 3720/10000 [4:26:33<7:45:17,  4.45s/it]
 37%|███▋      | 3721/10000 [4:26:37<7:49:23,  4.49s/it]
 37%|███▋      | 3722/10000 [4:26:42<7:53:31,  4.53s/it]
 37%|███▋      | 3723/10000 [4:26:46<7:53:07,  4.52s/it]
 37%|███▋      | 3724/10000 [4:26:50<7:43:44,  4.43s/it]
 37%|███▋      | 3725/10000 [4:26:55<7:46:15,  4.46s/it]
 37%|███▋      | 3726/10000 [4:26:59<7:39:11,  4.39s/it]
 37%|███▋      | 3727/10000 [4:27:04<7:43:19,  4.43s/it]
 37%|███▋      | 3728/10000 [4:27:08<7:36:56,  4.37s/it]
 37%|███▋      | 3729/10000 [4:27:12<7:39:03,  4.39s/it]
 37%|███▋      | 3730/10000 [4:27:17<7:34:14,  4.35s/it]
 37%|███▋      | 3731/10000 [4:27:21<7:41:00,  4.41s/it]
 37%|███▋      | 3732/10000 [4:27:25<7:31:49,  4.33s/it]
 37%|███▋      | 3733/10000 [4:27:30<7:38:52,  4.39s/it]
 37%|███▋      | 3734/10000 [4:27:34<7:35:34,  4.36s/it]
 37%|███▋      | 3735/10000 [4:27:39<7:41:33,  4.42s/it]
 37%|███▋      | 3736/10000 [4:27:43<7:36:58,  4.38s/it]
 37%|███▋      | 3737/10000 [4:27:47<7:40:27,  4.41s/it]
 37%|███▋      | 3738/10000 [4:27:52<7:44:46,  4.45s/it]
 37%|███▋      | 3739/10000 [4:27:57<7:48:07,  4.49s/it]
 37%|███▋      | 3740/10000 [4:28:01<7:47:52,  4.48s/it]
                                                        

 37%|███▋      | 3740/10000 [4:28:01<7:47:52,  4.48s/it]
 37%|███▋      | 3741/10000 [4:28:06<7:47:46,  4.48s/it]
 37%|███▋      | 3742/10000 [4:28:10<7:33:55,  4.35s/it]
 37%|███▋      | 3743/10000 [4:28:14<7:40:49,  4.42s/it]
 37%|███▋      | 3744/10000 [4:28:18<7:32:47,  4.34s/it]
 37%|███▋      | 3745/10000 [4:28:23<7:36:48,  4.38s/it]
 37%|███▋      | 3746/10000 [4:28:27<7:29:12,  4.31s/it]
 37%|███▋      | 3747/10000 [4:28:32<7:40:53,  4.42s/it]
 37%|███▋      | 3748/10000 [4:28:36<7:32:23,  4.34s/it]
 37%|███▋      | 3749/10000 [4:28:40<7:34:39,  4.36s/it]
 38%|███▊      | 3750/10000 [4:28:44<7:27:12,  4.29s/it]
 38%|███▊      | 3751/10000 [4:28:49<7:31:48,  4.34s/it]
 38%|███▊      | 3752/10000 [4:28:53<7:32:27,  4.35s/it]
 38%|███▊      | 3753/10000 [4:28:57<7:28:16,  4.31s/it]
 38%|███▊      | 3754/10000 [4:29:02<7:23:17,  4.26s/it]
 38%|███▊      | 3755/10000 [4:29:06<7:28:07,  4.31s/it]
 38%|███▊      | 3756/10000 [4:29:10<7:15:58,  4.19s/it]
 38%|███▊      | 3757/10000 [4:29:14<7:24:44,  4.27s/it]
 38%|███▊      | 3758/10000 [4:29:19<7:34:13,  4.37s/it]
 38%|███▊      | 3759/10000 [4:29:23<7:36:28,  4.39s/it]
 38%|███▊      | 3760/10000 [4:29:28<7:38:32,  4.41s/it]
                                                        

 38%|███▊      | 3760/10000 [4:29:28<7:38:32,  4.41s/it]
 38%|███▊      | 3761/10000 [4:29:32<7:30:49,  4.34s/it]
 38%|███▊      | 3762/10000 [4:29:36<7:18:58,  4.22s/it]
 38%|███▊      | 3763/10000 [4:29:40<7:19:26,  4.23s/it]
 38%|███▊      | 3764/10000 [4:29:44<7:17:43,  4.21s/it]
 38%|███▊      | 3765/10000 [4:29:49<7:26:14,  4.29s/it]
 38%|███▊      | 3766/10000 [4:29:53<7:22:36,  4.26s/it]
 38%|███▊      | 3767/10000 [4:29:57<7:29:00,  4.32s/it]
 38%|███▊      | 3768/10000 [4:30:02<7:30:38,  4.34s/it]
 38%|███▊      | 3769/10000 [4:30:06<7:27:12,  4.31s/it]
 38%|███▊      | 3770/10000 [4:30:11<7:37:50,  4.41s/it]
 38%|███▊      | 3771/10000 [4:30:15<7:30:04,  4.34s/it]
 38%|███▊      | 3772/10000 [4:30:19<7:32:32,  4.36s/it]
 38%|███▊      | 3773/10000 [4:30:23<7:18:10,  4.22s/it]
 38%|███▊      | 3774/10000 [4:30:28<7:23:47,  4.28s/it]
 38%|███▊      | 3775/10000 [4:30:32<7:18:21,  4.23s/it]
 38%|███▊      | 3776/10000 [4:30:36<7:23:34,  4.28s/it]
 38%|███▊      | 3777/10000 [4:30:41<7:41:08,  4.45s/it]
 38%|███▊      | 3778/10000 [4:30:45<7:39:52,  4.43s/it]
 38%|███▊      | 3779/10000 [4:30:49<7:30:12,  4.34s/it]
 38%|███▊      | 3780/10000 [4:30:54<7:24:50,  4.29s/it]
                                                        

 38%|███▊      | 3780/10000 [4:30:54<7:24:50,  4.29s/it]
 38%|███▊      | 3781/10000 [4:30:58<7:28:01,  4.32s/it]
 38%|███▊      | 3782/10000 [4:31:02<7:14:23,  4.19s/it]
 38%|███▊      | 3783/10000 [4:31:06<7:24:52,  4.29s/it]
 38%|███▊      | 3784/10000 [4:31:11<7:19:34,  4.24s/it]
 38%|███▊      | 3785/10000 [4:31:15<7:24:38,  4.29s/it]
 38%|███▊      | 3786/10000 [4:31:19<7:19:37,  4.24s/it]
 38%|███▊      | 3787/10000 [4:31:24<7:24:40,  4.29s/it]
 38%|███▊      | 3788/10000 [4:31:28<7:18:58,  4.24s/it]
 38%|███▊      | 3789/10000 [4:31:32<7:24:22,  4.29s/it]
 38%|███▊      | 3790/10000 [4:31:36<7:15:22,  4.21s/it]
 38%|███▊      | 3791/10000 [4:31:41<7:26:13,  4.31s/it]
 38%|███▊      | 3792/10000 [4:31:45<7:29:21,  4.34s/it]
 38%|███▊      | 3793/10000 [4:31:49<7:22:26,  4.28s/it]
 38%|███▊      | 3794/10000 [4:31:54<7:26:46,  4.32s/it]
 38%|███▊      | 3795/10000 [4:31:58<7:27:22,  4.33s/it]
 38%|███▊      | 3796/10000 [4:32:03<7:34:32,  4.40s/it]
 38%|███▊      | 3797/10000 [4:32:06<7:19:04,  4.25s/it]
 38%|███▊      | 3798/10000 [4:32:11<7:33:45,  4.39s/it]
 38%|███▊      | 3799/10000 [4:32:15<7:18:12,  4.24s/it]
 38%|███▊      | 3800/10000 [4:32:19<7:23:35,  4.29s/it]
                                                        
{'loss': '0.1443', 'grad_norm': '0.1654', 'learning_rate': '3.191e-06', 'num_tokens': '1.072e+06', 'completions/mean_length': '4.569', 'completions/min_length': '4.1', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.569', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2275', 'rewards/reward_semantic/std': '0.03527', 'rewards/reward_length/mean': '-0.02581', 'rewards/reward_length/std': '0.00853', 'reward': '1.067', 'reward_std': '0.01058', 'frac_reward_zero_std': '0.75', 'kl': '3.457', 'entropy': '0.1025', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.381', 'epoch': '0.5938'}
{'loss': '0.1368', 'grad_norm': '0.004967', 'learning_rate': '3.181e-06', 'num_tokens': '1.078e+06', 'completions/mean_length': '4.731', 'completions/min_length': '4.45', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.731', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3353', 'rewards/reward_semantic/std': '0.04486', 'rewards/reward_length/mean': '-0.03362', 'rewards/reward_length/std': '0.004612', 'reward': '1.099', 'reward_std': '0.01348', 'frac_reward_zero_std': '0.8', 'kl': '2.951', 'entropy': '0.08397', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.369', 'epoch': '0.5971'}
{'loss': '0.1225', 'grad_norm': '1.703e-06', 'learning_rate': '3.171e-06', 'num_tokens': '1.085e+06', 'completions/mean_length': '4.919', 'completions/min_length': '4.7', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.919', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2669', 'rewards/reward_semantic/std': '0.04899', 'rewards/reward_length/mean': '-0.02893', 'rewards/reward_length/std': '0.008699', 'reward': '1.079', 'reward_std': '0.01464', 'frac_reward_zero_std': '0.65', 'kl': '3.028', 'entropy': '0.09231', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.353', 'epoch': '0.6004'}
{'loss': '0.09691', 'grad_norm': '12.12', 'learning_rate': '3.161e-06', 'num_tokens': '1.09e+06', 'completions/mean_length': '4.4', 'completions/min_length': '4.25', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.4', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3429', 'rewards/reward_semantic/std': '0.05508', 'rewards/reward_length/mean': '-0.02693', 'rewards/reward_length/std': '0.008546', 'reward': '1.102', 'reward_std': '0.01627', 'frac_reward_zero_std': '0.75', 'kl': '2.854', 'entropy': '0.06867', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.321', 'epoch': '0.6037'}
{'loss': '0.08206', 'grad_norm': '0.003227', 'learning_rate': '3.151e-06', 'num_tokens': '1.096e+06', 'completions/mean_length': '4.844', 'completions/min_length': '4.35', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.844', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4347', 'rewards/reward_semantic/std': '0.02639', 'rewards/reward_length/mean': '-0.03367', 'rewards/reward_length/std': '0.005003', 'reward': '1.129', 'reward_std': '0.007824', 'frac_reward_zero_std': '0.75', 'kl': '2.345', 'entropy': '0.08205', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.436', 'epoch': '0.607'}
{'loss': '0.08856', 'grad_norm': '0.03304', 'learning_rate': '3.141e-06', 'num_tokens': '1.103e+06', 'completions/mean_length': '5.694', 'completions/min_length': '5.35', 'completions/max_length': '6.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.694', 'completions/min_terminated_length': '5.35', 'completions/max_terminated_length': '6.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.03557', 'rewards/reward_semantic/std': '0.04014', 'rewards/reward_length/mean': '-0.0238', 'rewards/reward_length/std': '0.004845', 'reward': '1.009', 'reward_std': '0.01203', 'frac_reward_zero_std': '0.75', 'kl': '2.76', 'entropy': '0.05862', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.496', 'epoch': '0.6102'}
{'loss': '0.1069', 'grad_norm': '9.488', 'learning_rate': '3.131e-06', 'num_tokens': '1.109e+06', 'completions/mean_length': '4.906', 'completions/min_length': '4.85', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.906', 'completions/min_terminated_length': '4.85', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3843', 'rewards/reward_semantic/std': '0.03775', 'rewards/reward_length/mean': '-0.03206', 'rewards/reward_length/std': '0.004348', 'reward': '1.114', 'reward_std': '0.01115', 'frac_reward_zero_std': '0.8', 'kl': '2.858', 'entropy': '0.1123', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.409', 'epoch': '0.6135'}
{'loss': '0.1188', 'grad_norm': '0.1183', 'learning_rate': '3.121e-06', 'num_tokens': '1.115e+06', 'completions/mean_length': '4.469', 'completions/min_length': '4.05', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.05', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2979', 'rewards/reward_semantic/std': '0.03719', 'rewards/reward_length/mean': '-0.02629', 'rewards/reward_length/std': '0.004777', 'reward': '1.088', 'reward_std': '0.01111', 'frac_reward_zero_std': '0.65', 'kl': '2.942', 'entropy': '0.1376', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.317', 'epoch': '0.6168'}
{'loss': '0.1502', 'grad_norm': '0.03997', 'learning_rate': '3.111e-06', 'num_tokens': '1.12e+06', 'completions/mean_length': '4.281', 'completions/min_length': '4', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.281', 'completions/min_terminated_length': '4', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4408', 'rewards/reward_semantic/std': '0.05581', 'rewards/reward_length/mean': '-0.025', 'rewards/reward_length/std': '0.0074', 'reward': '1.131', 'reward_std': '0.01681', 'frac_reward_zero_std': '0.7', 'kl': '3.008', 'entropy': '0.1223', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.275', 'epoch': '0.6201'}

 38%|███▊      | 3800/10000 [4:32:19<7:23:35,  4.29s/it]
 38%|███▊      | 3801/10000 [4:32:24<7:17:27,  4.23s/it]
 38%|███▊      | 3802/10000 [4:32:28<7:15:25,  4.22s/it]
 38%|███▊      | 3803/10000 [4:32:32<7:04:57,  4.11s/it]
 38%|███▊      | 3804/10000 [4:32:36<7:14:08,  4.20s/it]
 38%|███▊      | 3805/10000 [4:32:40<7:13:37,  4.20s/it]
 38%|███▊      | 3806/10000 [4:32:45<7:20:17,  4.26s/it]
 38%|███▊      | 3807/10000 [4:32:49<7:24:22,  4.31s/it]
 38%|███▊      | 3808/10000 [4:32:53<7:18:59,  4.25s/it]
 38%|███▊      | 3809/10000 [4:32:58<7:23:57,  4.30s/it]
 38%|███▊      | 3810/10000 [4:33:02<7:18:24,  4.25s/it]
 38%|███▊      | 3811/10000 [4:33:06<7:23:28,  4.30s/it]
 38%|███▊      | 3812/10000 [4:33:10<7:17:50,  4.25s/it]
 38%|███▊      | 3813/10000 [4:33:15<7:43:06,  4.49s/it]
 38%|███▊      | 3814/10000 [4:33:19<7:31:35,  4.38s/it]
 38%|███▊      | 3815/10000 [4:33:24<7:36:04,  4.42s/it]
 38%|███▊      | 3816/10000 [4:33:28<7:35:31,  4.42s/it]
 38%|███▊      | 3817/10000 [4:33:33<7:29:41,  4.36s/it]
 38%|███▊      | 3818/10000 [4:33:37<7:33:07,  4.40s/it]
 38%|███▊      | 3819/10000 [4:33:41<7:26:07,  4.33s/it]
 38%|███▊      | 3820/10000 [4:33:46<7:38:19,  4.45s/it]
                                                        

 38%|███▊      | 3820/10000 [4:33:46<7:38:19,  4.45s/it]
 38%|███▊      | 3821/10000 [4:33:50<7:31:39,  4.39s/it]
 38%|███▊      | 3822/10000 [4:33:55<7:35:35,  4.42s/it]
 38%|███▊      | 3823/10000 [4:33:59<7:27:16,  4.34s/it]
 38%|███▊      | 3824/10000 [4:34:03<7:32:44,  4.40s/it]
 38%|███▊      | 3825/10000 [4:34:08<7:27:32,  4.35s/it]
 38%|███▊      | 3826/10000 [4:34:12<7:35:17,  4.42s/it]
 38%|███▊      | 3827/10000 [4:34:17<7:34:21,  4.42s/it]
 38%|███▊      | 3828/10000 [4:34:21<7:37:47,  4.45s/it]
 38%|███▊      | 3829/10000 [4:34:26<7:39:48,  4.47s/it]
 38%|███▊      | 3830/10000 [4:34:30<7:32:14,  4.40s/it]
 38%|███▊      | 3831/10000 [4:34:34<7:36:05,  4.44s/it]
 38%|███▊      | 3832/10000 [4:34:39<7:29:07,  4.37s/it]
 38%|███▊      | 3833/10000 [4:34:44<7:47:28,  4.55s/it]
 38%|███▊      | 3834/10000 [4:34:48<7:38:16,  4.46s/it]
 38%|███▊      | 3835/10000 [4:34:52<7:38:03,  4.46s/it]
 38%|███▊      | 3836/10000 [4:34:56<7:29:20,  4.37s/it]
 38%|███▊      | 3837/10000 [4:35:01<7:30:08,  4.38s/it]
 38%|███▊      | 3838/10000 [4:35:05<7:26:38,  4.35s/it]
 38%|███▊      | 3839/10000 [4:35:10<7:28:53,  4.37s/it]
 38%|███▊      | 3840/10000 [4:35:14<7:29:52,  4.38s/it]
                                                        

 38%|███▊      | 3840/10000 [4:35:14<7:29:52,  4.38s/it]
 38%|███▊      | 3841/10000 [4:35:19<7:43:43,  4.52s/it]
 38%|███▊      | 3842/10000 [4:35:23<7:41:45,  4.50s/it]
 38%|███▊      | 3843/10000 [4:35:28<7:44:06,  4.52s/it]
 38%|███▊      | 3844/10000 [4:35:32<7:41:06,  4.49s/it]
 38%|███▊      | 3845/10000 [4:35:37<7:38:22,  4.47s/it]
 38%|███▊      | 3846/10000 [4:35:41<7:27:56,  4.37s/it]
 38%|███▊      | 3847/10000 [4:35:45<7:29:18,  4.38s/it]
 38%|███▊      | 3848/10000 [4:35:49<7:21:00,  4.30s/it]
 38%|███▊      | 3849/10000 [4:35:54<7:24:27,  4.34s/it]
 38%|███▊      | 3850/10000 [4:35:58<7:18:02,  4.27s/it]
 39%|███▊      | 3851/10000 [4:36:03<7:33:09,  4.42s/it]
 39%|███▊      | 3852/10000 [4:36:07<7:33:20,  4.42s/it]
 39%|███▊      | 3853/10000 [4:36:11<7:23:59,  4.33s/it]
 39%|███▊      | 3854/10000 [4:36:16<7:26:07,  4.36s/it]
 39%|███▊      | 3855/10000 [4:36:20<7:18:39,  4.28s/it]
 39%|███▊      | 3856/10000 [4:36:24<7:22:32,  4.32s/it]
 39%|███▊      | 3857/10000 [4:36:28<7:16:09,  4.26s/it]
 39%|███▊      | 3858/10000 [4:36:33<7:20:34,  4.30s/it]
 39%|███▊      | 3859/10000 [4:36:37<7:14:34,  4.25s/it]
 39%|███▊      | 3860/10000 [4:36:41<7:19:19,  4.29s/it]
                                                        

 39%|███▊      | 3860/10000 [4:36:41<7:19:19,  4.29s/it]
 39%|███▊      | 3861/10000 [4:36:45<7:06:33,  4.17s/it]
 39%|███▊      | 3862/10000 [4:36:49<7:14:17,  4.25s/it]
 39%|███▊      | 3863/10000 [4:36:54<7:10:45,  4.21s/it]
 39%|███▊      | 3864/10000 [4:36:58<7:16:51,  4.27s/it]
 39%|███▊      | 3865/10000 [4:37:02<7:20:13,  4.31s/it]
 39%|███▊      | 3866/10000 [4:37:07<7:16:23,  4.27s/it]
 39%|███▊      | 3867/10000 [4:37:11<7:13:04,  4.24s/it]
 39%|███▊      | 3868/10000 [4:37:15<7:09:10,  4.20s/it]
 39%|███▊      | 3869/10000 [4:37:19<7:14:29,  4.25s/it]
 39%|███▊      | 3870/10000 [4:37:23<7:10:29,  4.21s/it]
 39%|███▊      | 3871/10000 [4:37:28<7:16:32,  4.27s/it]
 39%|███▊      | 3872/10000 [4:37:32<7:18:17,  4.29s/it]
 39%|███▊      | 3873/10000 [4:37:37<7:32:20,  4.43s/it]
 39%|███▊      | 3874/10000 [4:37:41<7:22:34,  4.33s/it]
 39%|███▉      | 3875/10000 [4:37:45<7:24:25,  4.35s/it]
 39%|███▉      | 3876/10000 [4:37:50<7:23:47,  4.35s/it]
 39%|███▉      | 3877/10000 [4:37:54<7:16:38,  4.28s/it]
 39%|███▉      | 3878/10000 [4:37:58<7:20:18,  4.32s/it]
 39%|███▉      | 3879/10000 [4:38:02<7:14:27,  4.26s/it]
 39%|███▉      | 3880/10000 [4:38:07<7:18:42,  4.30s/it]
                                                        

 39%|███▉      | 3880/10000 [4:38:07<7:18:42,  4.30s/it]
 39%|███▉      | 3881/10000 [4:38:11<7:13:25,  4.25s/it]
 39%|███▉      | 3882/10000 [4:38:15<7:18:19,  4.30s/it]
 39%|███▉      | 3883/10000 [4:38:19<7:12:26,  4.24s/it]
 39%|███▉      | 3884/10000 [4:38:24<7:17:15,  4.29s/it]
 39%|███▉      | 3885/10000 [4:38:28<7:10:20,  4.22s/it]
 39%|███▉      | 3886/10000 [4:38:32<7:16:27,  4.28s/it]
 39%|███▉      | 3887/10000 [4:38:37<7:19:29,  4.31s/it]
 39%|███▉      | 3888/10000 [4:38:41<7:07:00,  4.19s/it]
 39%|███▉      | 3889/10000 [4:38:45<7:07:48,  4.20s/it]
 39%|███▉      | 3890/10000 [4:38:49<7:03:28,  4.16s/it]
 39%|███▉      | 3891/10000 [4:38:53<7:12:08,  4.24s/it]
 39%|███▉      | 3892/10000 [4:38:57<7:09:12,  4.22s/it]
 39%|███▉      | 3893/10000 [4:39:02<7:23:43,  4.36s/it]
 39%|███▉      | 3894/10000 [4:39:06<7:16:54,  4.29s/it]
 39%|███▉      | 3895/10000 [4:39:11<7:21:10,  4.34s/it]
 39%|███▉      | 3896/10000 [4:39:15<7:12:45,  4.25s/it]
 39%|███▉      | 3897/10000 [4:39:19<7:18:28,  4.31s/it]
 39%|███▉      | 3898/10000 [4:39:24<7:22:40,  4.35s/it]
 39%|███▉      | 3899/10000 [4:39:28<7:16:19,  4.29s/it]
 39%|███▉      | 3900/10000 [4:39:32<7:20:20,  4.33s/it]
                                                        

 39%|███▉      | 3900/10000 [4:39:32<7:20:20,  4.33s/it]
 39%|███▉      | 3901/10000 [4:39:36<7:14:18,  4.27s/it]
 39%|███▉      | 3902/10000 [4:39:41<7:20:38,  4.34s/it]
 39%|███▉      | 3903/10000 [4:39:45<7:14:40,  4.28s/it]
 39%|███▉      | 3904/10000 [4:39:49<7:20:16,  4.33s/it]
 39%|███▉      | 3905/10000 [4:39:54<7:13:20,  4.27s/it]
 39%|███▉      | 3906/10000 [4:39:58<7:18:31,  4.32s/it]
 39%|███▉      | 3907/10000 [4:40:02<7:13:11,  4.27s/it]
 39%|███▉      | 3908/10000 [4:40:07<7:17:59,  4.31s/it]
 39%|███▉      | 3909/10000 [4:40:11<7:12:44,  4.26s/it]
 39%|███▉      | 3910/10000 [4:40:15<7:17:56,  4.31s/it]
 39%|███▉      | 3911/10000 [4:40:20<7:21:18,  4.35s/it]
 39%|███▉      | 3912/10000 [4:40:24<7:14:30,  4.28s/it]
 39%|███▉      | 3913/10000 [4:40:28<7:17:58,  4.32s/it]
 39%|███▉      | 3914/10000 [4:40:32<7:11:37,  4.26s/it]
 39%|███▉      | 3915/10000 [4:40:37<7:17:04,  4.31s/it]
 39%|███▉      | 3916/10000 [4:40:41<7:11:08,  4.25s/it]
 39%|███▉      | 3917/10000 [4:40:45<7:15:46,  4.30s/it]
 39%|███▉      | 3918/10000 [4:40:50<7:17:58,  4.32s/it]
 39%|███▉      | 3919/10000 [4:40:54<7:18:27,  4.33s/it]
 39%|███▉      | 3920/10000 [4:40:58<7:11:15,  4.26s/it]
                                                        

 39%|███▉      | 3920/10000 [4:40:58<7:11:15,  4.26s/it]
 39%|███▉      | 3921/10000 [4:41:02<7:16:11,  4.31s/it]
 39%|███▉      | 3922/10000 [4:41:07<7:21:14,  4.36s/it]
 39%|███▉      | 3923/10000 [4:41:11<7:15:08,  4.30s/it]
 39%|███▉      | 3924/10000 [4:41:15<7:18:58,  4.33s/it]
 39%|███▉      | 3925/10000 [4:41:19<7:05:43,  4.20s/it]
 39%|███▉      | 3926/10000 [4:41:24<7:14:16,  4.29s/it]
 39%|███▉      | 3927/10000 [4:41:28<7:09:31,  4.24s/it]
 39%|███▉      | 3928/10000 [4:41:32<7:15:00,  4.30s/it]
 39%|███▉      | 3929/10000 [4:41:37<7:09:33,  4.25s/it]
 39%|███▉      | 3930/10000 [4:41:41<7:15:09,  4.30s/it]
 39%|███▉      | 3931/10000 [4:41:45<7:10:12,  4.25s/it]
 39%|███▉      | 3932/10000 [4:41:50<7:15:58,  4.31s/it]
 39%|███▉      | 3933/10000 [4:41:54<7:32:20,  4.47s/it]
 39%|███▉      | 3934/10000 [4:41:59<7:22:50,  4.38s/it]
 39%|███▉      | 3935/10000 [4:42:03<7:24:23,  4.40s/it]
 39%|███▉      | 3936/10000 [4:42:07<7:12:57,  4.28s/it]
 39%|███▉      | 3937/10000 [4:42:12<7:20:44,  4.36s/it]
 39%|███▉      | 3938/10000 [4:42:16<7:17:20,  4.33s/it]
 39%|███▉      | 3939/10000 [4:42:20<7:23:44,  4.39s/it]
 39%|███▉      | 3940/10000 [4:42:25<7:19:50,  4.35s/it]
                                                        

 39%|███▉      | 3940/10000 [4:42:25<7:19:50,  4.35s/it]
 39%|███▉      | 3941/10000 [4:42:29<7:25:21,  4.41s/it]
 39%|███▉      | 3942/10000 [4:42:33<7:15:33,  4.31s/it]
 39%|███▉      | 3943/10000 [4:42:38<7:21:44,  4.38s/it]
 39%|███▉      | 3944/10000 [4:42:42<7:10:36,  4.27s/it]
 39%|███▉      | 3945/10000 [4:42:46<7:18:35,  4.35s/it]
 39%|███▉      | 3946/10000 [4:42:51<7:15:23,  4.32s/it]
 39%|███▉      | 3947/10000 [4:42:55<7:26:47,  4.43s/it]
 39%|███▉      | 3948/10000 [4:43:00<7:29:20,  4.45s/it]
 39%|███▉      | 3949/10000 [4:43:04<7:23:09,  4.39s/it]
 40%|███▉      | 3950/10000 [4:43:09<7:31:44,  4.48s/it]
 40%|███▉      | 3951/10000 [4:43:13<7:25:01,  4.41s/it]
 40%|███▉      | 3952/10000 [4:43:18<7:29:08,  4.46s/it]
 40%|███▉      | 3953/10000 [4:43:22<7:22:54,  4.39s/it]
 40%|███▉      | 3954/10000 [4:43:26<7:31:30,  4.48s/it]
 40%|███▉      | 3955/10000 [4:43:31<7:24:28,  4.41s/it]
 40%|███▉      | 3956/10000 [4:43:35<7:28:47,  4.46s/it]
 40%|███▉      | 3957/10000 [4:43:40<7:23:01,  4.40s/it]
 40%|███▉      | 3958/10000 [4:43:44<7:27:16,  4.44s/it]
 40%|███▉      | 3959/10000 [4:43:49<7:29:28,  4.46s/it]
 40%|███▉      | 3960/10000 [4:43:53<7:23:06,  4.40s/it]
                                                        

 40%|███▉      | 3960/10000 [4:43:53<7:23:06,  4.40s/it]
 40%|███▉      | 3961/10000 [4:43:57<7:27:43,  4.45s/it]
 40%|███▉      | 3962/10000 [4:44:02<7:22:07,  4.39s/it]
 40%|███▉      | 3963/10000 [4:44:06<7:26:33,  4.44s/it]
 40%|███▉      | 3964/10000 [4:44:10<7:20:50,  4.38s/it]
 40%|███▉      | 3965/10000 [4:44:15<7:24:47,  4.42s/it]
 40%|███▉      | 3966/10000 [4:44:19<7:18:28,  4.36s/it]
 40%|███▉      | 3967/10000 [4:44:24<7:27:26,  4.45s/it]
 40%|███▉      | 3968/10000 [4:44:28<7:20:49,  4.38s/it]
 40%|███▉      | 3969/10000 [4:44:33<7:24:59,  4.43s/it]
 40%|███▉      | 3970/10000 [4:44:37<7:27:42,  4.45s/it]
 40%|███▉      | 3971/10000 [4:44:41<7:19:49,  4.38s/it]
 40%|███▉      | 3972/10000 [4:44:46<7:24:36,  4.43s/it]
 40%|███▉      | 3973/10000 [4:44:50<7:19:08,  4.37s/it]
 40%|███▉      | 3974/10000 [4:44:55<7:23:55,  4.42s/it]
 40%|███▉      | 3975/10000 [4:44:59<7:18:36,  4.37s/it]
 40%|███▉      | 3976/10000 [4:45:03<7:23:23,  4.42s/it]
 40%|███▉      | 3977/10000 [4:45:07<7:10:47,  4.29s/it]
 40%|███▉      | 3978/10000 [4:45:12<7:18:12,  4.37s/it]
 40%|███▉      | 3979/10000 [4:45:16<7:13:39,  4.32s/it]
 40%|███▉      | 3980/10000 [4:45:21<7:26:30,  4.45s/it]
                                                        
{'loss': '0.1426', 'grad_norm': '0.0005487', 'learning_rate': '3.101e-06', 'num_tokens': '1.126e+06', 'completions/mean_length': '4.469', 'completions/min_length': '4.1', 'completions/max_length': '4.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '4.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2858', 'rewards/reward_semantic/std': '0.06735', 'rewards/reward_length/mean': '-0.03271', 'rewards/reward_length/std': '0.006394', 'reward': '1.084', 'reward_std': '0.02032', 'frac_reward_zero_std': '0.7', 'kl': '3.328', 'entropy': '0.1113', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.27', 'epoch': '0.6234'}
{'loss': '0.113', 'grad_norm': '20.82', 'learning_rate': '3.091e-06', 'num_tokens': '1.132e+06', 'completions/mean_length': '4.531', 'completions/min_length': '4.25', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.531', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4083', 'rewards/reward_semantic/std': '0.01782', 'rewards/reward_length/mean': '-0.02544', 'rewards/reward_length/std': '0.00581', 'reward': '1.121', 'reward_std': '0.005493', 'frac_reward_zero_std': '0.75', 'kl': '3.016', 'entropy': '0.09024', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.307', 'epoch': '0.6266'}
{'loss': '0.1053', 'grad_norm': '0.01921', 'learning_rate': '3.08e-06', 'num_tokens': '1.138e+06', 'completions/mean_length': '4.931', 'completions/min_length': '4.55', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.931', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3411', 'rewards/reward_semantic/std': '0.03733', 'rewards/reward_length/mean': '-0.02854', 'rewards/reward_length/std': '0.005313', 'reward': '1.101', 'reward_std': '0.01125', 'frac_reward_zero_std': '0.7', 'kl': '2.559', 'entropy': '0.1149', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.382', 'epoch': '0.6299'}
{'loss': '0.1182', 'grad_norm': '0.0003437', 'learning_rate': '3.071e-06', 'num_tokens': '1.144e+06', 'completions/mean_length': '5.206', 'completions/min_length': '4.7', 'completions/max_length': '5.75', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.206', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.75', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3294', 'rewards/reward_semantic/std': '0.03547', 'rewards/reward_length/mean': '-0.03425', 'rewards/reward_length/std': '0.004579', 'reward': '1.097', 'reward_std': '0.01066', 'frac_reward_zero_std': '0.7', 'kl': '2.887', 'entropy': '0.06497', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.341', 'epoch': '0.6332'}
{'loss': '0.13', 'grad_norm': '0.1045', 'learning_rate': '3.061e-06', 'num_tokens': '1.149e+06', 'completions/mean_length': '4.688', 'completions/min_length': '4.45', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.688', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3018', 'rewards/reward_semantic/std': '0.005051', 'rewards/reward_length/mean': '-0.02861', 'rewards/reward_length/std': '0.005185', 'reward': '1.089', 'reward_std': '0.001623', 'frac_reward_zero_std': '0.85', 'kl': '2.974', 'entropy': '0.06907', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.261', 'epoch': '0.6365'}
{'loss': '0.1169', 'grad_norm': '0.0003512', 'learning_rate': '3.051e-06', 'num_tokens': '1.155e+06', 'completions/mean_length': '4.55', 'completions/min_length': '4.45', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.55', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4469', 'rewards/reward_semantic/std': '0.008839', 'rewards/reward_length/mean': '-0.03062', 'rewards/reward_length/std': '0.0006905', 'reward': '1.133', 'reward_std': '0.002617', 'frac_reward_zero_std': '0.95', 'kl': '3.138', 'entropy': '0.04055', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.257', 'epoch': '0.6398'}
{'loss': '0.09426', 'grad_norm': '0.03249', 'learning_rate': '3.04e-06', 'num_tokens': '1.161e+06', 'completions/mean_length': '4.881', 'completions/min_length': '4.65', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.881', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2973', 'rewards/reward_semantic/std': '0.02429', 'rewards/reward_length/mean': '-0.02185', 'rewards/reward_length/std': '0.004518', 'reward': '1.088', 'reward_std': '0.007138', 'frac_reward_zero_std': '0.85', 'kl': '3.008', 'entropy': '0.06128', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.269', 'epoch': '0.643'}
{'loss': '0.1001', 'grad_norm': '0.008626', 'learning_rate': '3.031e-06', 'num_tokens': '1.168e+06', 'completions/mean_length': '4.656', 'completions/min_length': '4.4', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.656', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2424', 'rewards/reward_semantic/std': '0.01799', 'rewards/reward_length/mean': '-0.02305', 'rewards/reward_length/std': '0.004001', 'reward': '1.072', 'reward_std': '0.005342', 'frac_reward_zero_std': '0.8', 'kl': '3.126', 'entropy': '0.0721', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.313', 'epoch': '0.6463'}
{'loss': '0.1798', 'grad_norm': '0.007288', 'learning_rate': '3.021e-06', 'num_tokens': '1.174e+06', 'completions/mean_length': '4.513', 'completions/min_length': '4.35', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.513', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3698', 'rewards/reward_semantic/std': '0.05892', 'rewards/reward_length/mean': '-0.0239', 'rewards/reward_length/std': '0.005285', 'reward': '1.11', 'reward_std': '0.01785', 'frac_reward_zero_std': '0.75', 'kl': '3.218', 'entropy': '0.07053', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.391', 'epoch': '0.6496'}

 40%|███▉      | 3980/10000 [4:45:21<7:26:30,  4.45s/it]
 40%|███▉      | 3981/10000 [4:45:26<7:29:36,  4.48s/it]
 40%|███▉      | 3982/10000 [4:45:30<7:26:16,  4.45s/it]
 40%|███▉      | 3983/10000 [4:45:34<7:28:45,  4.47s/it]
 40%|███▉      | 3984/10000 [4:45:39<7:20:48,  4.40s/it]
 40%|███▉      | 3985/10000 [4:45:43<7:25:01,  4.44s/it]
 40%|███▉      | 3986/10000 [4:45:48<7:23:43,  4.43s/it]
 40%|███▉      | 3987/10000 [4:45:52<7:27:09,  4.46s/it]
 40%|███▉      | 3988/10000 [4:45:56<7:20:50,  4.40s/it]
 40%|███▉      | 3989/10000 [4:46:01<7:25:11,  4.44s/it]
 40%|███▉      | 3990/10000 [4:46:05<7:27:06,  4.46s/it]
 40%|███▉      | 3991/10000 [4:46:10<7:20:31,  4.40s/it]
 40%|███▉      | 3992/10000 [4:46:14<7:24:54,  4.44s/it]
 40%|███▉      | 3993/10000 [4:46:18<7:18:59,  4.38s/it]
 40%|███▉      | 3994/10000 [4:46:24<7:38:42,  4.58s/it]
 40%|███▉      | 3995/10000 [4:46:28<7:26:11,  4.46s/it]
 40%|███▉      | 3996/10000 [4:46:32<7:28:59,  4.49s/it]
 40%|███▉      | 3997/10000 [4:46:36<7:21:14,  4.41s/it]
 40%|███▉      | 3998/10000 [4:46:41<7:26:15,  4.46s/it]
 40%|███▉      | 3999/10000 [4:46:45<7:19:43,  4.40s/it]
 40%|████      | 4000/10000 [4:46:50<7:24:00,  4.44s/it]
                                                        

 40%|████      | 4000/10000 [4:46:50<7:24:00,  4.44s/it]
 40%|████      | 4001/10000 [4:46:55<7:50:55,  4.71s/it]
 40%|████      | 4002/10000 [4:47:00<7:45:44,  4.66s/it]
 40%|████      | 4003/10000 [4:47:04<7:42:42,  4.63s/it]
 40%|████      | 4004/10000 [4:47:09<7:31:28,  4.52s/it]
 40%|████      | 4005/10000 [4:47:13<7:32:23,  4.53s/it]
 40%|████      | 4006/10000 [4:47:18<7:39:46,  4.60s/it]
 40%|████      | 4007/10000 [4:47:22<7:38:24,  4.59s/it]
 40%|████      | 4008/10000 [4:47:27<7:25:45,  4.46s/it]
 40%|████      | 4009/10000 [4:47:31<7:28:31,  4.49s/it]
 40%|████      | 4010/10000 [4:47:36<7:34:22,  4.55s/it]
 40%|████      | 4011/10000 [4:47:40<7:33:53,  4.55s/it]
 40%|████      | 4012/10000 [4:47:45<7:34:07,  4.55s/it]
 40%|████      | 4013/10000 [4:47:49<7:24:57,  4.46s/it]
 40%|████      | 4014/10000 [4:47:54<7:28:20,  4.49s/it]
 40%|████      | 4015/10000 [4:47:58<7:19:03,  4.40s/it]
 40%|████      | 4016/10000 [4:48:02<7:15:38,  4.37s/it]
 40%|████      | 4017/10000 [4:48:06<7:11:32,  4.33s/it]
 40%|████      | 4018/10000 [4:48:11<7:17:41,  4.39s/it]
 40%|████      | 4019/10000 [4:48:15<7:12:56,  4.34s/it]
 40%|████      | 4020/10000 [4:48:20<7:19:28,  4.41s/it]
                                                        

 40%|████      | 4020/10000 [4:48:20<7:19:28,  4.41s/it]
 40%|████      | 4021/10000 [4:48:24<7:14:55,  4.36s/it]
 40%|████      | 4022/10000 [4:48:29<7:20:33,  4.42s/it]
 40%|████      | 4023/10000 [4:48:33<7:23:57,  4.46s/it]
 40%|████      | 4024/10000 [4:48:37<7:17:00,  4.39s/it]
 40%|████      | 4025/10000 [4:48:42<7:24:55,  4.47s/it]
 40%|████      | 4026/10000 [4:48:46<7:17:54,  4.40s/it]
 40%|████      | 4027/10000 [4:48:51<7:17:35,  4.40s/it]
 40%|████      | 4028/10000 [4:48:55<7:12:59,  4.35s/it]
 40%|████      | 4029/10000 [4:48:59<7:18:11,  4.40s/it]
 40%|████      | 4030/10000 [4:49:04<7:13:20,  4.36s/it]
 40%|████      | 4031/10000 [4:49:08<7:27:48,  4.50s/it]
 40%|████      | 4032/10000 [4:49:13<7:20:21,  4.43s/it]
 40%|████      | 4033/10000 [4:49:18<7:32:32,  4.55s/it]
 40%|████      | 4034/10000 [4:49:22<7:31:00,  4.54s/it]
 40%|████      | 4035/10000 [4:49:26<7:19:16,  4.42s/it]
 40%|████      | 4036/10000 [4:49:31<7:21:06,  4.44s/it]
 40%|████      | 4037/10000 [4:49:35<7:14:25,  4.37s/it]
 40%|████      | 4038/10000 [4:49:39<7:19:32,  4.42s/it]
 40%|████      | 4039/10000 [4:49:44<7:15:52,  4.39s/it]
 40%|████      | 4040/10000 [4:49:48<7:19:41,  4.43s/it]
                                                        

 40%|████      | 4040/10000 [4:49:48<7:19:41,  4.43s/it]
 40%|████      | 4041/10000 [4:49:53<7:16:34,  4.40s/it]
 40%|████      | 4042/10000 [4:49:57<7:19:44,  4.43s/it]
 40%|████      | 4043/10000 [4:50:02<7:22:23,  4.46s/it]
 40%|████      | 4044/10000 [4:50:06<7:14:26,  4.38s/it]
 40%|████      | 4045/10000 [4:50:11<7:24:31,  4.48s/it]
 40%|████      | 4046/10000 [4:50:15<7:14:43,  4.38s/it]
 40%|████      | 4047/10000 [4:50:19<7:16:37,  4.40s/it]
 40%|████      | 4048/10000 [4:50:23<7:11:41,  4.35s/it]
 40%|████      | 4049/10000 [4:50:28<7:16:28,  4.40s/it]
 40%|████      | 4050/10000 [4:50:32<7:11:20,  4.35s/it]
 41%|████      | 4051/10000 [4:50:37<7:17:11,  4.41s/it]
 41%|████      | 4052/10000 [4:50:41<7:12:30,  4.36s/it]
 41%|████      | 4053/10000 [4:50:45<7:16:52,  4.41s/it]
 41%|████      | 4054/10000 [4:50:50<7:12:47,  4.37s/it]
 41%|████      | 4055/10000 [4:50:54<7:18:23,  4.42s/it]
 41%|████      | 4056/10000 [4:50:59<7:20:47,  4.45s/it]
 41%|████      | 4057/10000 [4:51:03<7:09:36,  4.34s/it]
 41%|████      | 4058/10000 [4:51:07<7:16:36,  4.41s/it]
 41%|████      | 4059/10000 [4:51:12<7:12:29,  4.37s/it]
 41%|████      | 4060/10000 [4:51:16<7:11:22,  4.36s/it]
                                                        

 41%|████      | 4060/10000 [4:51:16<7:11:22,  4.36s/it]
 41%|████      | 4061/10000 [4:51:20<7:08:33,  4.33s/it]
 41%|████      | 4062/10000 [4:51:25<7:14:54,  4.39s/it]
 41%|████      | 4063/10000 [4:51:29<7:11:07,  4.36s/it]
 41%|████      | 4064/10000 [4:51:34<7:17:36,  4.42s/it]
 41%|████      | 4065/10000 [4:51:38<7:20:28,  4.45s/it]
 41%|████      | 4066/10000 [4:51:42<7:13:50,  4.39s/it]
 41%|████      | 4067/10000 [4:51:47<7:19:48,  4.45s/it]
 41%|████      | 4068/10000 [4:51:51<7:14:13,  4.39s/it]
 41%|████      | 4069/10000 [4:51:56<7:18:02,  4.43s/it]
 41%|████      | 4070/10000 [4:52:00<7:14:25,  4.40s/it]
 41%|████      | 4071/10000 [4:52:05<7:17:54,  4.43s/it]
 41%|████      | 4072/10000 [4:52:09<7:29:32,  4.55s/it]
 41%|████      | 4073/10000 [4:52:14<7:20:26,  4.46s/it]
 41%|████      | 4074/10000 [4:52:18<7:22:15,  4.48s/it]
 41%|████      | 4075/10000 [4:52:23<7:15:31,  4.41s/it]
 41%|████      | 4076/10000 [4:52:27<7:19:37,  4.45s/it]
 41%|████      | 4077/10000 [4:52:31<7:13:31,  4.39s/it]
 41%|████      | 4078/10000 [4:52:36<7:11:54,  4.38s/it]
 41%|████      | 4079/10000 [4:52:40<7:09:30,  4.35s/it]
 41%|████      | 4080/10000 [4:52:44<7:14:37,  4.41s/it]
                                                        

 41%|████      | 4080/10000 [4:52:44<7:14:37,  4.41s/it]
 41%|████      | 4081/10000 [4:52:49<7:09:20,  4.35s/it]
 41%|████      | 4082/10000 [4:52:53<7:15:33,  4.42s/it]
 41%|████      | 4083/10000 [4:52:58<7:20:12,  4.46s/it]
 41%|████      | 4084/10000 [4:53:03<7:31:39,  4.58s/it]
 41%|████      | 4085/10000 [4:53:07<7:30:48,  4.57s/it]
 41%|████      | 4086/10000 [4:53:12<7:32:36,  4.59s/it]
 41%|████      | 4087/10000 [4:53:16<7:31:55,  4.59s/it]
 41%|████      | 4088/10000 [4:53:21<7:30:38,  4.57s/it]
 41%|████      | 4089/10000 [4:53:26<7:31:26,  4.58s/it]
 41%|████      | 4090/10000 [4:53:30<7:30:00,  4.57s/it]
 41%|████      | 4091/10000 [4:53:34<7:18:51,  4.46s/it]
 41%|████      | 4092/10000 [4:53:39<7:21:56,  4.49s/it]
 41%|████      | 4093/10000 [4:53:43<7:07:55,  4.35s/it]
 41%|████      | 4094/10000 [4:53:47<7:06:32,  4.33s/it]
 41%|████      | 4095/10000 [4:53:51<7:04:17,  4.31s/it]
 41%|████      | 4096/10000 [4:53:56<7:08:37,  4.36s/it]
 41%|████      | 4097/10000 [4:54:01<7:18:44,  4.46s/it]
 41%|████      | 4098/10000 [4:54:05<7:17:30,  4.45s/it]
 41%|████      | 4099/10000 [4:54:10<7:20:16,  4.48s/it]
 41%|████      | 4100/10000 [4:54:14<7:15:47,  4.43s/it]
                                                        

 41%|████      | 4100/10000 [4:54:14<7:15:47,  4.43s/it]
 41%|████      | 4101/10000 [4:54:19<7:33:18,  4.61s/it]
 41%|████      | 4102/10000 [4:54:23<7:22:34,  4.50s/it]
 41%|████      | 4103/10000 [4:54:28<7:38:19,  4.66s/it]
 41%|████      | 4104/10000 [4:54:33<7:33:19,  4.61s/it]
 41%|████      | 4105/10000 [4:54:37<7:22:48,  4.51s/it]
 41%|████      | 4106/10000 [4:54:42<7:24:57,  4.53s/it]
 41%|████      | 4107/10000 [4:54:46<7:17:53,  4.46s/it]
 41%|████      | 4108/10000 [4:54:50<7:20:10,  4.48s/it]
 41%|████      | 4109/10000 [4:54:55<7:22:38,  4.51s/it]
 41%|████      | 4110/10000 [4:54:59<7:09:06,  4.37s/it]
 41%|████      | 4111/10000 [4:55:04<7:13:52,  4.42s/it]
 41%|████      | 4112/10000 [4:55:08<7:08:19,  4.36s/it]
 41%|████      | 4113/10000 [4:55:12<7:12:48,  4.41s/it]
 41%|████      | 4114/10000 [4:55:17<7:08:34,  4.37s/it]
 41%|████      | 4115/10000 [4:55:21<7:13:51,  4.42s/it]
 41%|████      | 4116/10000 [4:55:26<7:13:00,  4.42s/it]
 41%|████      | 4117/10000 [4:55:30<7:17:15,  4.46s/it]
 41%|████      | 4118/10000 [4:55:35<7:15:25,  4.44s/it]
 41%|████      | 4119/10000 [4:55:39<7:18:49,  4.48s/it]
 41%|████      | 4120/10000 [4:55:44<7:20:44,  4.50s/it]
                                                        

 41%|████      | 4120/10000 [4:55:44<7:20:44,  4.50s/it]
 41%|████      | 4121/10000 [4:55:48<7:13:48,  4.43s/it]
 41%|████      | 4122/10000 [4:55:52<7:17:17,  4.46s/it]
 41%|████      | 4123/10000 [4:55:57<7:07:14,  4.36s/it]
 41%|████      | 4124/10000 [4:56:01<7:13:42,  4.43s/it]
 41%|████▏     | 4125/10000 [4:56:05<7:07:18,  4.36s/it]
 41%|████▏     | 4126/10000 [4:56:10<7:13:38,  4.43s/it]
 41%|████▏     | 4127/10000 [4:56:14<7:16:59,  4.46s/it]
 41%|████▏     | 4128/10000 [4:56:19<7:10:58,  4.40s/it]
 41%|████▏     | 4129/10000 [4:56:23<7:14:40,  4.44s/it]
 41%|████▏     | 4130/10000 [4:56:27<7:06:20,  4.36s/it]
 41%|████▏     | 4131/10000 [4:56:32<7:11:19,  4.41s/it]
 41%|████▏     | 4132/10000 [4:56:36<7:14:47,  4.45s/it]
 41%|████▏     | 4133/10000 [4:56:41<7:08:41,  4.38s/it]
 41%|████▏     | 4134/10000 [4:56:45<7:13:38,  4.44s/it]
 41%|████▏     | 4135/10000 [4:56:50<7:08:23,  4.38s/it]
 41%|████▏     | 4136/10000 [4:56:54<7:12:56,  4.43s/it]
 41%|████▏     | 4137/10000 [4:56:59<7:27:12,  4.58s/it]
 41%|████▏     | 4138/10000 [4:57:03<7:09:44,  4.40s/it]
 41%|████▏     | 4139/10000 [4:57:08<7:14:27,  4.45s/it]
 41%|████▏     | 4140/10000 [4:57:12<7:08:56,  4.39s/it]
                                                        

 41%|████▏     | 4140/10000 [4:57:12<7:08:56,  4.39s/it]
 41%|████▏     | 4141/10000 [4:57:16<7:13:53,  4.44s/it]
 41%|████▏     | 4142/10000 [4:57:21<7:10:51,  4.41s/it]
 41%|████▏     | 4143/10000 [4:57:25<7:04:27,  4.35s/it]
 41%|████▏     | 4144/10000 [4:57:30<7:11:40,  4.42s/it]
 41%|████▏     | 4145/10000 [4:57:35<7:33:59,  4.65s/it]
 41%|████▏     | 4146/10000 [4:57:39<7:23:32,  4.55s/it]
 41%|████▏     | 4147/10000 [4:57:43<7:15:31,  4.46s/it]
 41%|████▏     | 4148/10000 [4:57:47<7:04:50,  4.36s/it]
 41%|████▏     | 4149/10000 [4:57:52<7:08:24,  4.39s/it]
 42%|████▏     | 4150/10000 [4:57:56<7:02:03,  4.33s/it]
 42%|████▏     | 4151/10000 [4:58:01<7:06:12,  4.37s/it]
 42%|████▏     | 4152/10000 [4:58:05<7:15:00,  4.46s/it]
 42%|████▏     | 4153/10000 [4:58:09<7:06:45,  4.38s/it]
 42%|████▏     | 4154/10000 [4:58:14<7:13:59,  4.45s/it]
 42%|████▏     | 4155/10000 [4:58:18<6:59:09,  4.30s/it]
 42%|████▏     | 4156/10000 [4:58:22<7:03:56,  4.35s/it]
 42%|████▏     | 4157/10000 [4:58:27<7:07:16,  4.39s/it]
 42%|████▏     | 4158/10000 [4:58:31<7:01:08,  4.33s/it]
 42%|████▏     | 4159/10000 [4:58:36<7:05:19,  4.37s/it]
 42%|████▏     | 4160/10000 [4:58:40<7:08:02,  4.40s/it]
                                                        
{'loss': '0.1322', 'grad_norm': '15.95', 'learning_rate': '3.011e-06', 'num_tokens': '1.179e+06', 'completions/mean_length': '4.825', 'completions/min_length': '4.8', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.825', 'completions/min_terminated_length': '4.8', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3245', 'rewards/reward_semantic/std': '0.05059', 'rewards/reward_length/mean': '-0.02803', 'rewards/reward_length/std': '0.005341', 'reward': '1.096', 'reward_std': '0.01521', 'frac_reward_zero_std': '0.75', 'kl': '3.109', 'entropy': '0.07227', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.384', 'epoch': '0.6529'}
{'loss': '0.105', 'grad_norm': '4.236e-05', 'learning_rate': '3e-06', 'num_tokens': '1.185e+06', 'completions/mean_length': '5.088', 'completions/min_length': '5', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.088', 'completions/min_terminated_length': '5', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2125', 'rewards/reward_semantic/std': '0.01035', 'rewards/reward_length/mean': '-0.03687', 'rewards/reward_length/std': '0.002991', 'reward': '1.062', 'reward_std': '0.003184', 'frac_reward_zero_std': '0.85', 'kl': '2.565', 'entropy': '0.04009', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.424', 'epoch': '0.6562'}
{'loss': '0.1317', 'grad_norm': '9.239', 'learning_rate': '2.99e-06', 'num_tokens': '1.191e+06', 'completions/mean_length': '4.975', 'completions/min_length': '4.65', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.975', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1627', 'rewards/reward_semantic/std': '0.0006313', 'rewards/reward_length/mean': '-0.03452', 'rewards/reward_length/std': '0.002561', 'reward': '1.047', 'reward_std': '0.0003118', 'frac_reward_zero_std': '0.75', 'kl': '3.039', 'entropy': '0.09135', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.424', 'epoch': '0.6594'}
{'loss': '0.1325', 'grad_norm': '0.0008369', 'learning_rate': '2.981e-06', 'num_tokens': '1.197e+06', 'completions/mean_length': '4.75', 'completions/min_length': '4.65', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.75', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2556', 'rewards/reward_semantic/std': '0.03049', 'rewards/reward_length/mean': '-0.0311', 'rewards/reward_length/std': '0.005351', 'reward': '1.075', 'reward_std': '0.009009', 'frac_reward_zero_std': '0.8', 'kl': '2.999', 'entropy': '0.05526', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.406', 'epoch': '0.6627'}
{'loss': '0.1119', 'grad_norm': '0.07549', 'learning_rate': '2.971e-06', 'num_tokens': '1.203e+06', 'completions/mean_length': '4.581', 'completions/min_length': '4.5', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.581', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.32', 'rewards/reward_semantic/std': '0', 'rewards/reward_length/mean': '-0.02961', 'rewards/reward_length/std': '0.001999', 'reward': '1.095', 'reward_std': '9.997e-05', 'frac_reward_zero_std': '0.85', 'kl': '2.794', 'entropy': '0.07048', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.368', 'epoch': '0.666'}
{'loss': '0.1157', 'grad_norm': '14.05', 'learning_rate': '2.96e-06', 'num_tokens': '1.21e+06', 'completions/mean_length': '4.469', 'completions/min_length': '3.95', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4044', 'rewards/reward_semantic/std': '0.0635', 'rewards/reward_length/mean': '-0.03127', 'rewards/reward_length/std': '0.006706', 'reward': '1.12', 'reward_std': '0.01901', 'frac_reward_zero_std': '0.55', 'kl': '3.105', 'entropy': '0.1089', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.401', 'epoch': '0.6693'}
{'loss': '0.1041', 'grad_norm': '21.41', 'learning_rate': '2.95e-06', 'num_tokens': '1.216e+06', 'completions/mean_length': '4.906', 'completions/min_length': '4.6', 'completions/max_length': '5.4', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.906', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3376', 'rewards/reward_semantic/std': '0.05316', 'rewards/reward_length/mean': '-0.03323', 'rewards/reward_length/std': '0.006004', 'reward': '1.1', 'reward_std': '0.01597', 'frac_reward_zero_std': '0.7', 'kl': '2.822', 'entropy': '0.1471', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.453', 'epoch': '0.6726'}
{'loss': '0.1174', 'grad_norm': '0.02733', 'learning_rate': '2.941e-06', 'num_tokens': '1.222e+06', 'completions/mean_length': '5.062', 'completions/min_length': '4.7', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.062', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1568', 'rewards/reward_semantic/std': '0.03327', 'rewards/reward_length/mean': '-0.0324', 'rewards/reward_length/std': '0.008782', 'reward': '1.045', 'reward_std': '0.009985', 'frac_reward_zero_std': '0.65', 'kl': '3.157', 'entropy': '0.1351', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.464', 'epoch': '0.6759'}
{'loss': '0.1384', 'grad_norm': '0.002373', 'learning_rate': '2.931e-06', 'num_tokens': '1.228e+06', 'completions/mean_length': '4.713', 'completions/min_length': '4.65', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.713', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.25', 'rewards/reward_semantic/std': '0', 'rewards/reward_length/mean': '-0.02583', 'rewards/reward_length/std': '0.001811', 'reward': '1.074', 'reward_std': '9.053e-05', 'frac_reward_zero_std': '0.95', 'kl': '3.188', 'entropy': '0.06173', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.39', 'epoch': '0.6791'}

 42%|████▏     | 4160/10000 [4:58:40<7:08:02,  4.40s/it]
 42%|████▏     | 4161/10000 [4:58:45<7:12:09,  4.44s/it]
 42%|████▏     | 4162/10000 [4:58:49<7:21:30,  4.54s/it]
 42%|████▏     | 4163/10000 [4:58:54<7:14:29,  4.47s/it]
 42%|████▏     | 4164/10000 [4:58:58<7:25:22,  4.58s/it]
 42%|████▏     | 4165/10000 [4:59:03<7:26:23,  4.59s/it]
 42%|████▏     | 4166/10000 [4:59:08<7:29:42,  4.62s/it]
 42%|████▏     | 4167/10000 [4:59:12<7:25:27,  4.58s/it]
 42%|████▏     | 4168/10000 [4:59:16<7:06:49,  4.39s/it]
 42%|████▏     | 4169/10000 [4:59:21<7:09:17,  4.42s/it]
 42%|████▏     | 4170/10000 [4:59:25<7:11:17,  4.44s/it]
 42%|████▏     | 4171/10000 [4:59:29<7:02:28,  4.35s/it]
 42%|████▏     | 4172/10000 [4:59:34<7:21:33,  4.55s/it]
 42%|████▏     | 4173/10000 [4:59:39<7:21:31,  4.55s/it]
 42%|████▏     | 4174/10000 [4:59:43<7:13:36,  4.47s/it]
 42%|████▏     | 4175/10000 [4:59:48<7:17:09,  4.50s/it]
 42%|████▏     | 4176/10000 [4:59:52<7:11:32,  4.45s/it]
 42%|████▏     | 4177/10000 [4:59:56<7:08:07,  4.41s/it]
 42%|████▏     | 4178/10000 [5:00:01<7:13:02,  4.46s/it]
 42%|████▏     | 4179/10000 [5:00:05<7:07:18,  4.40s/it]
 42%|████▏     | 4180/10000 [5:00:10<7:17:22,  4.51s/it]
                                                        

 42%|████▏     | 4180/10000 [5:00:10<7:17:22,  4.51s/it]
 42%|████▏     | 4181/10000 [5:00:14<7:07:32,  4.41s/it]
 42%|████▏     | 4182/10000 [5:00:19<7:07:44,  4.41s/it]
 42%|████▏     | 4183/10000 [5:00:23<7:08:08,  4.42s/it]
 42%|████▏     | 4184/10000 [5:00:27<7:01:44,  4.35s/it]
 42%|████▏     | 4185/10000 [5:00:32<7:03:58,  4.37s/it]
 42%|████▏     | 4186/10000 [5:00:36<7:11:11,  4.45s/it]
 42%|████▏     | 4187/10000 [5:00:40<7:02:11,  4.36s/it]
 42%|████▏     | 4188/10000 [5:00:45<7:04:54,  4.39s/it]
 42%|████▏     | 4189/10000 [5:00:49<6:57:20,  4.31s/it]
 42%|████▏     | 4190/10000 [5:00:53<7:02:06,  4.36s/it]
 42%|████▏     | 4191/10000 [5:00:58<7:08:31,  4.43s/it]
 42%|████▏     | 4192/10000 [5:01:02<7:07:57,  4.42s/it]
 42%|████▏     | 4193/10000 [5:01:07<7:09:06,  4.43s/it]
 42%|████▏     | 4194/10000 [5:01:11<6:54:23,  4.28s/it]
 42%|████▏     | 4195/10000 [5:01:15<6:59:01,  4.33s/it]
 42%|████▏     | 4196/10000 [5:01:20<7:04:26,  4.39s/it]
 42%|████▏     | 4197/10000 [5:01:24<6:56:38,  4.31s/it]
 42%|████▏     | 4198/10000 [5:01:28<7:00:14,  4.35s/it]
 42%|████▏     | 4199/10000 [5:01:32<6:53:59,  4.28s/it]
 42%|████▏     | 4200/10000 [5:01:37<6:56:09,  4.31s/it]
                                                        

 42%|████▏     | 4200/10000 [5:01:37<6:56:09,  4.31s/it]
 42%|████▏     | 4201/10000 [5:01:41<6:59:44,  4.34s/it]
 42%|████▏     | 4202/10000 [5:01:46<7:05:43,  4.41s/it]
 42%|████▏     | 4203/10000 [5:01:50<7:06:01,  4.41s/it]
 42%|████▏     | 4204/10000 [5:01:55<7:06:40,  4.42s/it]
 42%|████▏     | 4205/10000 [5:01:59<6:56:31,  4.31s/it]
 42%|████▏     | 4206/10000 [5:02:03<7:00:25,  4.35s/it]
 42%|████▏     | 4207/10000 [5:02:08<7:04:34,  4.40s/it]
 42%|████▏     | 4208/10000 [5:02:12<7:08:50,  4.44s/it]
 42%|████▏     | 4209/10000 [5:02:17<7:18:01,  4.54s/it]
 42%|████▏     | 4210/10000 [5:02:22<7:18:27,  4.54s/it]
 42%|████▏     | 4211/10000 [5:02:26<7:12:16,  4.48s/it]
 42%|████▏     | 4212/10000 [5:02:30<7:09:21,  4.45s/it]
 42%|████▏     | 4213/10000 [5:02:35<7:14:48,  4.51s/it]
 42%|████▏     | 4214/10000 [5:02:39<7:16:17,  4.52s/it]
 42%|████▏     | 4215/10000 [5:02:44<7:08:54,  4.45s/it]
 42%|████▏     | 4216/10000 [5:02:48<7:12:00,  4.48s/it]
 42%|████▏     | 4217/10000 [5:02:53<7:14:33,  4.51s/it]
 42%|████▏     | 4218/10000 [5:02:58<7:33:26,  4.71s/it]
 42%|████▏     | 4219/10000 [5:03:02<7:25:50,  4.63s/it]
 42%|████▏     | 4220/10000 [5:03:07<7:25:13,  4.62s/it]
                                                        

 42%|████▏     | 4220/10000 [5:03:07<7:25:13,  4.62s/it]
 42%|████▏     | 4221/10000 [5:03:11<7:13:02,  4.50s/it]
 42%|████▏     | 4222/10000 [5:03:16<7:12:54,  4.50s/it]
 42%|████▏     | 4223/10000 [5:03:20<7:04:00,  4.40s/it]
 42%|████▏     | 4224/10000 [5:03:24<7:04:48,  4.41s/it]
 42%|████▏     | 4225/10000 [5:03:29<7:07:56,  4.45s/it]
 42%|████▏     | 4226/10000 [5:03:33<7:02:14,  4.39s/it]
 42%|████▏     | 4227/10000 [5:03:38<7:06:42,  4.43s/it]
 42%|████▏     | 4228/10000 [5:03:42<7:01:50,  4.39s/it]
 42%|████▏     | 4229/10000 [5:03:47<7:08:56,  4.46s/it]
 42%|████▏     | 4230/10000 [5:03:51<7:12:07,  4.49s/it]
 42%|████▏     | 4231/10000 [5:03:55<7:04:59,  4.42s/it]
 42%|████▏     | 4232/10000 [5:04:00<7:08:12,  4.45s/it]
 42%|████▏     | 4233/10000 [5:04:04<6:59:52,  4.37s/it]
 42%|████▏     | 4234/10000 [5:04:09<7:04:46,  4.42s/it]
 42%|████▏     | 4235/10000 [5:04:13<7:08:25,  4.46s/it]
 42%|████▏     | 4236/10000 [5:04:18<7:10:33,  4.48s/it]
 42%|████▏     | 4237/10000 [5:04:22<7:06:45,  4.44s/it]
 42%|████▏     | 4238/10000 [5:04:26<6:55:53,  4.33s/it]
 42%|████▏     | 4239/10000 [5:04:31<7:06:56,  4.45s/it]
 42%|████▏     | 4240/10000 [5:04:36<7:10:30,  4.48s/it]
                                                        

 42%|████▏     | 4240/10000 [5:04:36<7:10:30,  4.48s/it]
 42%|████▏     | 4241/10000 [5:04:40<7:03:25,  4.41s/it]
 42%|████▏     | 4242/10000 [5:04:44<7:06:35,  4.45s/it]
 42%|████▏     | 4243/10000 [5:04:49<7:02:41,  4.41s/it]
 42%|████▏     | 4244/10000 [5:04:53<6:57:06,  4.35s/it]
 42%|████▏     | 4245/10000 [5:04:57<6:58:59,  4.37s/it]
 42%|████▏     | 4246/10000 [5:05:01<6:44:59,  4.22s/it]
 42%|████▏     | 4247/10000 [5:05:06<6:50:27,  4.28s/it]
 42%|████▏     | 4248/10000 [5:05:10<6:53:56,  4.32s/it]
 42%|████▏     | 4249/10000 [5:05:14<6:56:44,  4.35s/it]
 42%|████▎     | 4250/10000 [5:05:19<7:01:14,  4.40s/it]
 43%|████▎     | 4251/10000 [5:05:23<6:53:48,  4.32s/it]
 43%|████▎     | 4252/10000 [5:05:27<6:56:22,  4.35s/it]
 43%|████▎     | 4253/10000 [5:05:32<7:01:58,  4.41s/it]
 43%|████▎     | 4254/10000 [5:05:36<7:05:37,  4.44s/it]
 43%|████▎     | 4255/10000 [5:05:41<7:12:51,  4.52s/it]
 43%|████▎     | 4256/10000 [5:05:45<7:01:37,  4.40s/it]
 43%|████▎     | 4257/10000 [5:05:50<7:02:36,  4.42s/it]
 43%|████▎     | 4258/10000 [5:05:54<7:11:16,  4.51s/it]
 43%|████▎     | 4259/10000 [5:05:59<7:10:30,  4.50s/it]
 43%|████▎     | 4260/10000 [5:06:04<7:12:37,  4.52s/it]
                                                        

 43%|████▎     | 4260/10000 [5:06:04<7:12:37,  4.52s/it]
 43%|████▎     | 4261/10000 [5:06:08<7:10:11,  4.50s/it]
 43%|████▎     | 4262/10000 [5:06:12<6:59:45,  4.39s/it]
 43%|████▎     | 4263/10000 [5:06:17<7:01:52,  4.41s/it]
 43%|████▎     | 4264/10000 [5:06:20<6:47:46,  4.27s/it]
 43%|████▎     | 4265/10000 [5:06:25<6:52:40,  4.32s/it]
 43%|████▎     | 4266/10000 [5:06:29<6:56:23,  4.36s/it]
 43%|████▎     | 4267/10000 [5:06:34<6:50:45,  4.30s/it]
 43%|████▎     | 4268/10000 [5:06:39<7:11:17,  4.51s/it]
 43%|████▎     | 4269/10000 [5:06:43<7:00:48,  4.41s/it]
 43%|████▎     | 4270/10000 [5:06:47<7:08:02,  4.48s/it]
 43%|████▎     | 4271/10000 [5:06:52<7:07:36,  4.48s/it]
 43%|████▎     | 4272/10000 [5:06:56<6:51:35,  4.31s/it]
 43%|████▎     | 4273/10000 [5:07:00<6:57:29,  4.37s/it]
 43%|████▎     | 4274/10000 [5:07:05<7:04:00,  4.44s/it]
 43%|████▎     | 4275/10000 [5:07:09<6:57:34,  4.38s/it]
 43%|████▎     | 4276/10000 [5:07:14<7:03:13,  4.44s/it]
 43%|████▎     | 4277/10000 [5:07:18<6:58:29,  4.39s/it]
 43%|████▎     | 4278/10000 [5:07:22<6:59:32,  4.40s/it]
 43%|████▎     | 4279/10000 [5:07:27<7:07:35,  4.48s/it]
 43%|████▎     | 4280/10000 [5:07:31<6:58:47,  4.39s/it]
                                                        

 43%|████▎     | 4280/10000 [5:07:31<6:58:47,  4.39s/it]
 43%|████▎     | 4281/10000 [5:07:36<6:59:46,  4.40s/it]
 43%|████▎     | 4282/10000 [5:07:40<6:51:53,  4.32s/it]
 43%|████▎     | 4283/10000 [5:07:44<6:55:00,  4.36s/it]
 43%|████▎     | 4284/10000 [5:07:48<6:48:53,  4.29s/it]
 43%|████▎     | 4285/10000 [5:07:53<6:50:46,  4.31s/it]
 43%|████▎     | 4286/10000 [5:07:57<6:59:59,  4.41s/it]
 43%|████▎     | 4287/10000 [5:08:01<6:45:36,  4.26s/it]
 43%|████▎     | 4288/10000 [5:08:06<6:50:18,  4.31s/it]
 43%|████▎     | 4289/10000 [5:08:10<6:45:30,  4.26s/it]
 43%|████▎     | 4290/10000 [5:08:14<6:48:12,  4.29s/it]
 43%|████▎     | 4291/10000 [5:08:19<6:52:15,  4.33s/it]
 43%|████▎     | 4292/10000 [5:08:23<6:46:44,  4.28s/it]
 43%|████▎     | 4293/10000 [5:08:27<6:51:05,  4.32s/it]
 43%|████▎     | 4294/10000 [5:08:31<6:45:42,  4.27s/it]
 43%|████▎     | 4295/10000 [5:08:36<6:50:16,  4.31s/it]
 43%|████▎     | 4296/10000 [5:08:40<6:53:36,  4.35s/it]
 43%|████▎     | 4297/10000 [5:08:44<6:47:40,  4.29s/it]
 43%|████▎     | 4298/10000 [5:08:49<7:02:28,  4.45s/it]
 43%|████▎     | 4299/10000 [5:08:53<6:47:32,  4.29s/it]
 43%|████▎     | 4300/10000 [5:08:58<6:50:52,  4.33s/it]
                                                        

 43%|████▎     | 4300/10000 [5:08:58<6:50:52,  4.33s/it]
 43%|████▎     | 4301/10000 [5:09:02<6:59:10,  4.41s/it]
 43%|████▎     | 4302/10000 [5:09:06<6:51:22,  4.33s/it]
 43%|████▎     | 4303/10000 [5:09:11<6:55:10,  4.37s/it]
 43%|████▎     | 4304/10000 [5:09:16<7:10:45,  4.54s/it]
 43%|████▎     | 4305/10000 [5:09:20<6:59:24,  4.42s/it]
 43%|████▎     | 4306/10000 [5:09:24<6:59:41,  4.42s/it]
 43%|████▎     | 4307/10000 [5:09:29<7:05:41,  4.49s/it]
 43%|████▎     | 4308/10000 [5:09:33<7:04:12,  4.47s/it]
 43%|████▎     | 4309/10000 [5:09:38<7:02:41,  4.46s/it]
 43%|████▎     | 4310/10000 [5:09:42<6:54:06,  4.37s/it]
 43%|████▎     | 4311/10000 [5:09:46<6:55:53,  4.39s/it]
 43%|████▎     | 4312/10000 [5:09:50<6:44:26,  4.27s/it]
 43%|████▎     | 4313/10000 [5:09:55<6:54:20,  4.37s/it]
 43%|████▎     | 4314/10000 [5:09:59<6:55:47,  4.39s/it]
 43%|████▎     | 4315/10000 [5:10:04<6:49:12,  4.32s/it]
 43%|████▎     | 4316/10000 [5:10:08<6:50:04,  4.33s/it]
 43%|████▎     | 4317/10000 [5:10:12<6:46:35,  4.29s/it]
 43%|████▎     | 4318/10000 [5:10:16<6:41:56,  4.24s/it]
 43%|████▎     | 4319/10000 [5:10:21<6:46:50,  4.30s/it]
 43%|████▎     | 4320/10000 [5:10:25<6:52:22,  4.36s/it]
                                                        

 43%|████▎     | 4320/10000 [5:10:25<6:52:22,  4.36s/it]
 43%|████▎     | 4321/10000 [5:10:30<6:54:14,  4.38s/it]
 43%|████▎     | 4322/10000 [5:10:34<6:55:54,  4.39s/it]
 43%|████▎     | 4323/10000 [5:10:38<6:48:24,  4.32s/it]
 43%|████▎     | 4324/10000 [5:10:43<6:51:37,  4.35s/it]
 43%|████▎     | 4325/10000 [5:10:47<6:58:23,  4.42s/it]
 43%|████▎     | 4326/10000 [5:10:52<6:58:17,  4.42s/it]
 43%|████▎     | 4327/10000 [5:10:56<6:52:04,  4.36s/it]
 43%|████▎     | 4328/10000 [5:11:00<6:39:52,  4.23s/it]
 43%|████▎     | 4329/10000 [5:11:04<6:39:53,  4.23s/it]
 43%|████▎     | 4330/10000 [5:11:08<6:35:43,  4.19s/it]
 43%|████▎     | 4331/10000 [5:11:13<6:49:35,  4.34s/it]
 43%|████▎     | 4332/10000 [5:11:17<6:46:20,  4.30s/it]
 43%|████▎     | 4333/10000 [5:11:21<6:35:33,  4.19s/it]
 43%|████▎     | 4334/10000 [5:11:25<6:43:04,  4.27s/it]
 43%|████▎     | 4335/10000 [5:11:29<6:39:55,  4.24s/it]
 43%|████▎     | 4336/10000 [5:11:34<6:46:26,  4.31s/it]
 43%|████▎     | 4337/10000 [5:11:38<6:51:01,  4.35s/it]
 43%|████▎     | 4338/10000 [5:11:43<6:45:43,  4.30s/it]
 43%|████▎     | 4339/10000 [5:11:47<6:53:02,  4.38s/it]
 43%|████▎     | 4340/10000 [5:11:51<6:46:47,  4.31s/it]
                                                        
{'loss': '0.1319', 'grad_norm': '2.542e-06', 'learning_rate': '2.92e-06', 'num_tokens': '1.234e+06', 'completions/mean_length': '4.494', 'completions/min_length': '4.25', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.494', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.341', 'rewards/reward_semantic/std': '0.03272', 'rewards/reward_length/mean': '-0.03428', 'rewards/reward_length/std': '0.006406', 'reward': '1.101', 'reward_std': '0.009873', 'frac_reward_zero_std': '0.8', 'kl': '3.05', 'entropy': '0.1013', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.39', 'epoch': '0.6824'}
{'loss': '0.1225', 'grad_norm': '9.653', 'learning_rate': '2.91e-06', 'num_tokens': '1.24e+06', 'completions/mean_length': '5.05', 'completions/min_length': '4.55', 'completions/max_length': '5.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.05', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2279', 'rewards/reward_semantic/std': '0.05897', 'rewards/reward_length/mean': '-0.03494', 'rewards/reward_length/std': '0.01029', 'reward': '1.067', 'reward_std': '0.01769', 'frac_reward_zero_std': '0.6', 'kl': '2.807', 'entropy': '0.178', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.479', 'epoch': '0.6857'}
{'loss': '0.1334', 'grad_norm': '0.0007522', 'learning_rate': '2.901e-06', 'num_tokens': '1.246e+06', 'completions/mean_length': '4.469', 'completions/min_length': '4.1', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.469', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3692', 'rewards/reward_semantic/std': '0.05005', 'rewards/reward_length/mean': '-0.03613', 'rewards/reward_length/std': '0.008074', 'reward': '1.109', 'reward_std': '0.01528', 'frac_reward_zero_std': '0.7', 'kl': '2.513', 'entropy': '0.107', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.325', 'epoch': '0.689'}
{'loss': '0.09828', 'grad_norm': '20.26', 'learning_rate': '2.891e-06', 'num_tokens': '1.252e+06', 'completions/mean_length': '5.112', 'completions/min_length': '4.4', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.112', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3063', 'rewards/reward_semantic/std': '0.1067', 'rewards/reward_length/mean': '-0.03962', 'rewards/reward_length/std': '0.007609', 'reward': '1.09', 'reward_std': '0.03206', 'frac_reward_zero_std': '0.5', 'kl': '2.746', 'entropy': '0.1641', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.494', 'epoch': '0.6923'}
{'loss': '0.1332', 'grad_norm': '7.507', 'learning_rate': '2.88e-06', 'num_tokens': '1.258e+06', 'completions/mean_length': '4.706', 'completions/min_length': '4.2', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.706', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4044', 'rewards/reward_semantic/std': '0.08523', 'rewards/reward_length/mean': '-0.03716', 'rewards/reward_length/std': '0.004724', 'reward': '1.119', 'reward_std': '0.0256', 'frac_reward_zero_std': '0.6', 'kl': '2.739', 'entropy': '0.1072', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.401', 'epoch': '0.6955'}
{'loss': '0.1043', 'grad_norm': '18.12', 'learning_rate': '2.871e-06', 'num_tokens': '1.264e+06', 'completions/mean_length': '4.688', 'completions/min_length': '4.1', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.688', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2485', 'rewards/reward_semantic/std': '0.1005', 'rewards/reward_length/mean': '-0.03767', 'rewards/reward_length/std': '0.01306', 'reward': '1.073', 'reward_std': '0.03022', 'frac_reward_zero_std': '0.55', 'kl': '3.024', 'entropy': '0.2518', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.382', 'epoch': '0.6988'}
{'loss': '0.1114', 'grad_norm': '0.0213', 'learning_rate': '2.861e-06', 'num_tokens': '1.27e+06', 'completions/mean_length': '4.338', 'completions/min_length': '3.9', 'completions/max_length': '4.8', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.338', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '4.8', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2521', 'rewards/reward_semantic/std': '0.06134', 'rewards/reward_length/mean': '-0.03264', 'rewards/reward_length/std': '0.007178', 'reward': '1.074', 'reward_std': '0.01832', 'frac_reward_zero_std': '0.6', 'kl': '2.713', 'entropy': '0.205', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.367', 'epoch': '0.7021'}
{'loss': '0.1261', 'grad_norm': '18.5', 'learning_rate': '2.851e-06', 'num_tokens': '1.276e+06', 'completions/mean_length': '4.381', 'completions/min_length': '4.2', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.381', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.6521', 'rewards/reward_semantic/std': '0.0444', 'rewards/reward_length/mean': '-0.0356', 'rewards/reward_length/std': '0.003573', 'reward': '1.194', 'reward_std': '0.01344', 'frac_reward_zero_std': '0.8', 'kl': '3.052', 'entropy': '0.1349', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.295', 'epoch': '0.7054'}
{'loss': '0.1458', 'grad_norm': '5.403', 'learning_rate': '2.841e-06', 'num_tokens': '1.283e+06', 'completions/mean_length': '4.956', 'completions/min_length': '4.45', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.956', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2299', 'rewards/reward_semantic/std': '0.06287', 'rewards/reward_length/mean': '-0.03884', 'rewards/reward_length/std': '0.00868', 'reward': '1.067', 'reward_std': '0.01926', 'frac_reward_zero_std': '0.5', 'kl': '3.249', 'entropy': '0.2328', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.362', 'epoch': '0.7087'}

 43%|████▎     | 4340/10000 [5:11:51<6:46:47,  4.31s/it]
 43%|████▎     | 4341/10000 [5:11:56<6:51:06,  4.36s/it]
 43%|████▎     | 4342/10000 [5:12:00<6:54:21,  4.39s/it]
 43%|████▎     | 4343/10000 [5:12:04<6:48:14,  4.33s/it]
 43%|████▎     | 4344/10000 [5:12:09<6:51:10,  4.36s/it]
 43%|████▎     | 4345/10000 [5:12:13<6:38:31,  4.23s/it]
 43%|████▎     | 4346/10000 [5:12:17<6:42:40,  4.27s/it]
 43%|████▎     | 4347/10000 [5:12:21<6:41:06,  4.26s/it]
 43%|████▎     | 4348/10000 [5:12:26<6:38:31,  4.23s/it]
 43%|████▎     | 4349/10000 [5:12:30<6:46:59,  4.32s/it]
 44%|████▎     | 4350/10000 [5:12:34<6:42:47,  4.28s/it]
 44%|████▎     | 4351/10000 [5:12:39<6:47:18,  4.33s/it]
 44%|████▎     | 4352/10000 [5:12:43<6:44:32,  4.30s/it]
 44%|████▎     | 4353/10000 [5:12:47<6:48:41,  4.34s/it]
 44%|████▎     | 4354/10000 [5:12:52<6:54:38,  4.41s/it]
 44%|████▎     | 4355/10000 [5:12:56<6:51:42,  4.38s/it]
 44%|████▎     | 4356/10000 [5:13:00<6:47:51,  4.34s/it]
 44%|████▎     | 4357/10000 [5:13:05<6:51:18,  4.37s/it]
 44%|████▎     | 4358/10000 [5:13:09<6:45:58,  4.32s/it]
 44%|████▎     | 4359/10000 [5:13:14<6:49:54,  4.36s/it]
 44%|████▎     | 4360/10000 [5:13:18<6:44:43,  4.31s/it]
                                                        

 44%|████▎     | 4360/10000 [5:13:18<6:44:43,  4.31s/it]
 44%|████▎     | 4361/10000 [5:13:22<6:49:19,  4.36s/it]
 44%|████▎     | 4362/10000 [5:13:27<6:52:48,  4.39s/it]
 44%|████▎     | 4363/10000 [5:13:31<6:56:49,  4.44s/it]
 44%|████▎     | 4364/10000 [5:13:36<6:57:01,  4.44s/it]
 44%|████▎     | 4365/10000 [5:13:40<6:49:03,  4.36s/it]
 44%|████▎     | 4366/10000 [5:13:44<6:51:23,  4.38s/it]
 44%|████▎     | 4367/10000 [5:13:49<6:52:59,  4.40s/it]
 44%|████▎     | 4368/10000 [5:13:53<6:45:55,  4.32s/it]
 44%|████▎     | 4369/10000 [5:13:57<6:48:44,  4.36s/it]
 44%|████▎     | 4370/10000 [5:14:02<6:52:21,  4.39s/it]
 44%|████▎     | 4371/10000 [5:14:06<6:46:48,  4.34s/it]
 44%|████▎     | 4372/10000 [5:14:10<6:50:33,  4.38s/it]
 44%|████▎     | 4373/10000 [5:14:15<6:44:01,  4.31s/it]
 44%|████▎     | 4374/10000 [5:14:19<6:44:10,  4.31s/it]
 44%|████▍     | 4375/10000 [5:14:24<6:53:18,  4.41s/it]
 44%|████▍     | 4376/10000 [5:14:28<6:49:47,  4.37s/it]
 44%|████▍     | 4377/10000 [5:14:32<6:55:30,  4.43s/it]
 44%|████▍     | 4378/10000 [5:14:37<6:49:15,  4.37s/it]
 44%|████▍     | 4379/10000 [5:14:41<6:53:06,  4.41s/it]
 44%|████▍     | 4380/10000 [5:14:46<7:06:54,  4.56s/it]
                                                        

 44%|████▍     | 4380/10000 [5:14:46<7:06:54,  4.56s/it]
 44%|████▍     | 4381/10000 [5:14:50<6:54:18,  4.42s/it]
 44%|████▍     | 4382/10000 [5:14:55<6:56:03,  4.44s/it]
 44%|████▍     | 4383/10000 [5:14:59<6:52:49,  4.41s/it]
 44%|████▍     | 4384/10000 [5:15:04<7:02:29,  4.51s/it]
 44%|████▍     | 4385/10000 [5:15:08<7:06:35,  4.56s/it]
 44%|████▍     | 4386/10000 [5:15:13<6:55:51,  4.44s/it]
 44%|████▍     | 4387/10000 [5:15:17<6:50:02,  4.38s/it]
 44%|████▍     | 4388/10000 [5:15:22<7:00:07,  4.49s/it]
 44%|████▍     | 4389/10000 [5:15:26<6:51:11,  4.40s/it]
 44%|████▍     | 4390/10000 [5:15:30<6:56:23,  4.45s/it]
 44%|████▍     | 4391/10000 [5:15:35<6:52:00,  4.41s/it]
 44%|████▍     | 4392/10000 [5:15:39<6:53:43,  4.43s/it]
 44%|████▍     | 4393/10000 [5:15:43<6:45:54,  4.34s/it]
 44%|████▍     | 4394/10000 [5:15:48<6:54:55,  4.44s/it]
 44%|████▍     | 4395/10000 [5:15:52<6:53:35,  4.43s/it]
 44%|████▍     | 4396/10000 [5:15:57<6:50:24,  4.39s/it]
 44%|████▍     | 4397/10000 [5:16:01<6:51:58,  4.41s/it]
 44%|████▍     | 4398/10000 [5:16:06<6:56:14,  4.46s/it]
 44%|████▍     | 4399/10000 [5:16:10<6:49:08,  4.38s/it]
 44%|████▍     | 4400/10000 [5:16:14<6:51:34,  4.41s/it]
                                                        

 44%|████▍     | 4400/10000 [5:16:14<6:51:34,  4.41s/it]
 44%|████▍     | 4401/10000 [5:16:19<6:53:37,  4.43s/it]
 44%|████▍     | 4402/10000 [5:16:24<7:22:09,  4.74s/it]
 44%|████▍     | 4403/10000 [5:16:29<7:14:04,  4.65s/it]
 44%|████▍     | 4404/10000 [5:16:33<7:00:12,  4.51s/it]
 44%|████▍     | 4405/10000 [5:16:37<6:58:37,  4.49s/it]
 44%|████▍     | 4406/10000 [5:16:42<6:57:38,  4.48s/it]
 44%|████▍     | 4407/10000 [5:16:46<6:48:21,  4.38s/it]
 44%|████▍     | 4408/10000 [5:16:50<6:51:43,  4.42s/it]
 44%|████▍     | 4409/10000 [5:16:55<6:47:02,  4.37s/it]
 44%|████▍     | 4410/10000 [5:16:59<6:49:23,  4.39s/it]
 44%|████▍     | 4411/10000 [5:17:04<7:00:53,  4.52s/it]
 44%|████▍     | 4412/10000 [5:17:08<6:51:31,  4.42s/it]
 44%|████▍     | 4413/10000 [5:17:13<6:52:55,  4.43s/it]
 44%|████▍     | 4414/10000 [5:17:17<6:51:19,  4.42s/it]
 44%|████▍     | 4415/10000 [5:17:21<6:53:06,  4.44s/it]
 44%|████▍     | 4416/10000 [5:17:26<6:54:46,  4.46s/it]
 44%|████▍     | 4417/10000 [5:17:30<6:47:29,  4.38s/it]
 44%|████▍     | 4418/10000 [5:17:35<6:50:42,  4.41s/it]
 44%|████▍     | 4419/10000 [5:17:39<6:52:39,  4.44s/it]
 44%|████▍     | 4420/10000 [5:17:43<6:45:45,  4.36s/it]
                                                        

 44%|████▍     | 4420/10000 [5:17:43<6:45:45,  4.36s/it]
 44%|████▍     | 4421/10000 [5:17:48<6:48:26,  4.39s/it]
 44%|████▍     | 4422/10000 [5:17:52<6:41:03,  4.31s/it]
 44%|████▍     | 4423/10000 [5:17:56<6:47:57,  4.39s/it]
 44%|████▍     | 4424/10000 [5:18:01<6:53:52,  4.45s/it]
 44%|████▍     | 4425/10000 [5:18:05<6:45:27,  4.36s/it]
 44%|████▍     | 4426/10000 [5:18:10<6:49:34,  4.41s/it]
 44%|████▍     | 4427/10000 [5:18:14<6:36:47,  4.27s/it]
 44%|████▍     | 4428/10000 [5:18:18<6:42:32,  4.33s/it]
 44%|████▍     | 4429/10000 [5:18:22<6:39:36,  4.30s/it]
 44%|████▍     | 4430/10000 [5:18:27<6:42:28,  4.34s/it]
 44%|████▍     | 4431/10000 [5:18:31<6:44:56,  4.36s/it]
 44%|████▍     | 4432/10000 [5:18:35<6:40:18,  4.31s/it]
 44%|████▍     | 4433/10000 [5:18:40<6:45:17,  4.37s/it]
 44%|████▍     | 4434/10000 [5:18:44<6:49:06,  4.41s/it]
 44%|████▍     | 4435/10000 [5:18:49<6:43:06,  4.35s/it]
 44%|████▍     | 4436/10000 [5:18:53<6:47:22,  4.39s/it]
 44%|████▍     | 4437/10000 [5:18:57<6:35:35,  4.27s/it]
 44%|████▍     | 4438/10000 [5:19:02<6:44:48,  4.37s/it]
 44%|████▍     | 4439/10000 [5:19:06<6:48:53,  4.41s/it]
 44%|████▍     | 4440/10000 [5:19:10<6:42:58,  4.35s/it]
                                                        

 44%|████▍     | 4440/10000 [5:19:10<6:42:58,  4.35s/it]
 44%|████▍     | 4441/10000 [5:19:15<6:47:34,  4.40s/it]
 44%|████▍     | 4442/10000 [5:19:19<6:42:10,  4.34s/it]
 44%|████▍     | 4443/10000 [5:19:24<7:00:42,  4.54s/it]
 44%|████▍     | 4444/10000 [5:19:29<7:03:55,  4.58s/it]
 44%|████▍     | 4445/10000 [5:19:33<6:53:37,  4.47s/it]
 44%|████▍     | 4446/10000 [5:19:37<6:46:41,  4.39s/it]
 44%|████▍     | 4447/10000 [5:19:42<6:52:19,  4.46s/it]
 44%|████▍     | 4448/10000 [5:19:46<6:44:41,  4.37s/it]
 44%|████▍     | 4449/10000 [5:19:51<6:47:13,  4.40s/it]
 44%|████▍     | 4450/10000 [5:19:55<6:41:04,  4.34s/it]
 45%|████▍     | 4451/10000 [5:19:59<6:44:29,  4.37s/it]
 45%|████▍     | 4452/10000 [5:20:03<6:42:44,  4.36s/it]
 45%|████▍     | 4453/10000 [5:20:08<6:35:42,  4.28s/it]
 45%|████▍     | 4454/10000 [5:20:13<6:56:35,  4.51s/it]
 45%|████▍     | 4455/10000 [5:20:17<6:47:42,  4.41s/it]
 45%|████▍     | 4456/10000 [5:20:21<6:46:58,  4.40s/it]
 45%|████▍     | 4457/10000 [5:20:26<6:50:50,  4.45s/it]
 45%|████▍     | 4458/10000 [5:20:30<6:44:22,  4.38s/it]
 45%|████▍     | 4459/10000 [5:20:34<6:40:29,  4.34s/it]
 45%|████▍     | 4460/10000 [5:20:38<6:35:43,  4.29s/it]
                                                        

 45%|████▍     | 4460/10000 [5:20:38<6:35:43,  4.29s/it]
 45%|████▍     | 4461/10000 [5:20:43<6:40:19,  4.34s/it]
 45%|████▍     | 4462/10000 [5:20:47<6:43:43,  4.37s/it]
 45%|████▍     | 4463/10000 [5:20:51<6:38:41,  4.32s/it]
 45%|████▍     | 4464/10000 [5:20:56<6:44:07,  4.38s/it]
 45%|████▍     | 4465/10000 [5:21:00<6:46:36,  4.41s/it]
 45%|████▍     | 4466/10000 [5:21:05<6:44:34,  4.39s/it]
 45%|████▍     | 4467/10000 [5:21:09<6:47:10,  4.42s/it]
 45%|████▍     | 4468/10000 [5:21:14<6:41:15,  4.35s/it]
 45%|████▍     | 4469/10000 [5:21:18<6:43:58,  4.38s/it]
 45%|████▍     | 4470/10000 [5:21:22<6:46:08,  4.41s/it]
 45%|████▍     | 4471/10000 [5:21:26<6:32:36,  4.26s/it]
 45%|████▍     | 4472/10000 [5:21:31<6:37:58,  4.32s/it]
 45%|████▍     | 4473/10000 [5:21:35<6:33:33,  4.27s/it]
 45%|████▍     | 4474/10000 [5:21:39<6:38:28,  4.33s/it]
 45%|████▍     | 4475/10000 [5:21:44<6:42:01,  4.37s/it]
 45%|████▍     | 4476/10000 [5:21:48<6:36:14,  4.30s/it]
 45%|████▍     | 4477/10000 [5:21:52<6:40:15,  4.35s/it]
 45%|████▍     | 4478/10000 [5:21:57<6:43:03,  4.38s/it]
 45%|████▍     | 4479/10000 [5:22:01<6:37:27,  4.32s/it]
 45%|████▍     | 4480/10000 [5:22:06<6:45:06,  4.40s/it]
                                                        

 45%|████▍     | 4480/10000 [5:22:06<6:45:06,  4.40s/it]
 45%|████▍     | 4481/10000 [5:22:10<6:50:24,  4.46s/it]
 45%|████▍     | 4482/10000 [5:22:15<6:44:06,  4.39s/it]
 45%|████▍     | 4483/10000 [5:22:19<6:49:21,  4.45s/it]
 45%|████▍     | 4484/10000 [5:22:24<6:56:32,  4.53s/it]
 45%|████▍     | 4485/10000 [5:22:28<6:50:09,  4.46s/it]
 45%|████▍     | 4486/10000 [5:22:33<6:58:12,  4.55s/it]
 45%|████▍     | 4487/10000 [5:22:37<6:58:19,  4.55s/it]
 45%|████▍     | 4488/10000 [5:22:42<6:50:19,  4.47s/it]
 45%|████▍     | 4489/10000 [5:22:46<6:52:16,  4.49s/it]
 45%|████▍     | 4490/10000 [5:22:51<6:45:09,  4.41s/it]
 45%|████▍     | 4491/10000 [5:22:55<6:50:30,  4.47s/it]
 45%|████▍     | 4492/10000 [5:23:00<6:50:26,  4.47s/it]
 45%|████▍     | 4493/10000 [5:23:04<6:44:45,  4.41s/it]
 45%|████▍     | 4494/10000 [5:23:09<6:54:05,  4.51s/it]
 45%|████▍     | 4495/10000 [5:23:13<6:48:35,  4.45s/it]
 45%|████▍     | 4496/10000 [5:23:17<6:46:42,  4.43s/it]
 45%|████▍     | 4497/10000 [5:23:22<6:48:31,  4.45s/it]
 45%|████▍     | 4498/10000 [5:23:26<6:44:46,  4.41s/it]
 45%|████▍     | 4499/10000 [5:23:31<6:46:32,  4.43s/it]
 45%|████▌     | 4500/10000 [5:23:36<7:02:11,  4.61s/it]
                                                        

 45%|████▌     | 4500/10000 [5:23:36<7:02:11,  4.61s/it]
 45%|████▌     | 4501/10000 [5:23:41<7:24:28,  4.85s/it]
 45%|████▌     | 4502/10000 [5:23:46<7:14:12,  4.74s/it]
 45%|████▌     | 4503/10000 [5:23:50<7:08:59,  4.68s/it]
 45%|████▌     | 4504/10000 [5:23:55<7:07:51,  4.67s/it]
 45%|████▌     | 4505/10000 [5:23:59<6:57:00,  4.55s/it]
 45%|████▌     | 4506/10000 [5:24:04<6:58:24,  4.57s/it]
 45%|████▌     | 4507/10000 [5:24:08<6:59:01,  4.58s/it]
 45%|████▌     | 4508/10000 [5:24:13<7:05:48,  4.65s/it]
 45%|████▌     | 4509/10000 [5:24:18<7:05:02,  4.64s/it]
 45%|████▌     | 4510/10000 [5:24:22<7:03:06,  4.62s/it]
 45%|████▌     | 4511/10000 [5:24:27<6:53:34,  4.52s/it]
 45%|████▌     | 4512/10000 [5:24:31<6:55:00,  4.54s/it]
 45%|████▌     | 4513/10000 [5:24:36<7:06:11,  4.66s/it]
 45%|████▌     | 4514/10000 [5:24:40<6:49:37,  4.48s/it]
 45%|████▌     | 4515/10000 [5:24:45<6:50:16,  4.49s/it]
 45%|████▌     | 4516/10000 [5:24:49<6:49:54,  4.48s/it]
 45%|████▌     | 4517/10000 [5:24:53<6:41:49,  4.40s/it]
 45%|████▌     | 4518/10000 [5:24:58<6:43:35,  4.42s/it]
 45%|████▌     | 4519/10000 [5:25:03<6:54:48,  4.54s/it]
 45%|████▌     | 4520/10000 [5:25:07<6:45:02,  4.43s/it]
                                                        
{'loss': '0.142', 'grad_norm': '0.0007442', 'learning_rate': '2.831e-06', 'num_tokens': '1.288e+06', 'completions/mean_length': '4.344', 'completions/min_length': '4.15', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.344', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5198', 'rewards/reward_semantic/std': '0.0322', 'rewards/reward_length/mean': '-0.03774', 'rewards/reward_length/std': '0.004153', 'reward': '1.154', 'reward_std': '0.009752', 'frac_reward_zero_std': '0.8', 'kl': '2.931', 'entropy': '0.09309', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.29', 'epoch': '0.7119'}
{'loss': '0.1147', 'grad_norm': '0.0006509', 'learning_rate': '2.821e-06', 'num_tokens': '1.294e+06', 'completions/mean_length': '4.306', 'completions/min_length': '4.25', 'completions/max_length': '4.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.306', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4244', 'rewards/reward_semantic/std': '0.03359', 'rewards/reward_length/mean': '-0.03252', 'rewards/reward_length/std': '0.001354', 'reward': '1.126', 'reward_std': '0.01006', 'frac_reward_zero_std': '0.85', 'kl': '3.078', 'entropy': '0.1004', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.304', 'epoch': '0.7152'}
{'loss': '0.1451', 'grad_norm': '7.773', 'learning_rate': '2.811e-06', 'num_tokens': '1.3e+06', 'completions/mean_length': '4.706', 'completions/min_length': '4.45', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.706', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4394', 'rewards/reward_semantic/std': '0.01647', 'rewards/reward_length/mean': '-0.03091', 'rewards/reward_length/std': '0.004325', 'reward': '1.13', 'reward_std': '0.005138', 'frac_reward_zero_std': '0.75', 'kl': '2.815', 'entropy': '0.09735', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.396', 'epoch': '0.7185'}
{'loss': '0.1062', 'grad_norm': '6.294e-07', 'learning_rate': '2.801e-06', 'num_tokens': '1.306e+06', 'completions/mean_length': '4.644', 'completions/min_length': '4.25', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.644', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4244', 'rewards/reward_semantic/std': '0.1068', 'rewards/reward_length/mean': '-0.04114', 'rewards/reward_length/std': '0.01021', 'reward': '1.125', 'reward_std': '0.03207', 'frac_reward_zero_std': '0.45', 'kl': '2.933', 'entropy': '0.1587', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.395', 'epoch': '0.7218'}
{'loss': '0.08211', 'grad_norm': '3.816e-05', 'learning_rate': '2.791e-06', 'num_tokens': '1.312e+06', 'completions/mean_length': '5.412', 'completions/min_length': '4.9', 'completions/max_length': '5.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.412', 'completions/min_terminated_length': '4.9', 'completions/max_terminated_length': '5.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2663', 'rewards/reward_semantic/std': '0.05606', 'rewards/reward_length/mean': '-0.03184', 'rewards/reward_length/std': '0.008807', 'reward': '1.078', 'reward_std': '0.01683', 'frac_reward_zero_std': '0.7', 'kl': '2.549', 'entropy': '0.1381', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.433', 'epoch': '0.7251'}
{'loss': '0.1463', 'grad_norm': '0.002145', 'learning_rate': '2.781e-06', 'num_tokens': '1.318e+06', 'completions/mean_length': '4.412', 'completions/min_length': '4.2', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.412', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4258', 'rewards/reward_semantic/std': '0.01275', 'rewards/reward_length/mean': '-0.03147', 'rewards/reward_length/std': '0.002631', 'reward': '1.126', 'reward_std': '0.003947', 'frac_reward_zero_std': '0.8', 'kl': '2.871', 'entropy': '0.1439', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.337', 'epoch': '0.7283'}
{'loss': '0.1188', 'grad_norm': '0.005623', 'learning_rate': '2.771e-06', 'num_tokens': '1.324e+06', 'completions/mean_length': '4.494', 'completions/min_length': '4.1', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.494', 'completions/min_terminated_length': '4.1', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3853', 'rewards/reward_semantic/std': '0.03432', 'rewards/reward_length/mean': '-0.04492', 'rewards/reward_length/std': '0.005854', 'reward': '1.113', 'reward_std': '0.01045', 'frac_reward_zero_std': '0.7', 'kl': '3.266', 'entropy': '0.1461', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.377', 'epoch': '0.7316'}
{'loss': '0.125', 'grad_norm': '0.001846', 'learning_rate': '2.761e-06', 'num_tokens': '1.33e+06', 'completions/mean_length': '4.787', 'completions/min_length': '4.65', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.787', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2781', 'rewards/reward_semantic/std': '0.03146', 'rewards/reward_length/mean': '-0.02119', 'rewards/reward_length/std': '0.001292', 'reward': '1.082', 'reward_std': '0.00941', 'frac_reward_zero_std': '0.9', 'kl': '3.277', 'entropy': '0.05567', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.348', 'epoch': '0.7349'}
{'loss': '0.1294', 'grad_norm': '0.04609', 'learning_rate': '2.751e-06', 'num_tokens': '1.336e+06', 'completions/mean_length': '4.7', 'completions/min_length': '4.4', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.7', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3835', 'rewards/reward_semantic/std': '0.08331', 'rewards/reward_length/mean': '-0.03755', 'rewards/reward_length/std': '0.01008', 'reward': '1.113', 'reward_std': '0.02507', 'frac_reward_zero_std': '0.65', 'kl': '2.582', 'entropy': '0.1944', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.475', 'epoch': '0.7382'}

 45%|████▌     | 4520/10000 [5:25:07<6:45:02,  4.43s/it]
 45%|████▌     | 4521/10000 [5:25:11<6:45:52,  4.44s/it]
 45%|████▌     | 4522/10000 [5:25:16<6:47:25,  4.46s/it]
 45%|████▌     | 4523/10000 [5:25:20<6:39:54,  4.38s/it]
 45%|████▌     | 4524/10000 [5:25:24<6:43:12,  4.42s/it]
 45%|████▌     | 4525/10000 [5:25:29<6:55:04,  4.55s/it]
 45%|████▌     | 4526/10000 [5:25:33<6:44:24,  4.43s/it]
 45%|████▌     | 4527/10000 [5:25:39<7:05:04,  4.66s/it]
 45%|████▌     | 4528/10000 [5:25:43<6:59:31,  4.60s/it]
 45%|████▌     | 4529/10000 [5:25:48<6:54:38,  4.55s/it]
 45%|████▌     | 4530/10000 [5:25:52<6:44:14,  4.43s/it]
 45%|████▌     | 4531/10000 [5:25:56<6:46:23,  4.46s/it]
 45%|████▌     | 4532/10000 [5:26:00<6:39:28,  4.38s/it]
 45%|████▌     | 4533/10000 [5:26:06<7:00:03,  4.61s/it]
 45%|████▌     | 4534/10000 [5:26:10<6:56:33,  4.57s/it]
 45%|████▌     | 4535/10000 [5:26:14<6:53:26,  4.54s/it]
 45%|████▌     | 4536/10000 [5:26:19<6:43:20,  4.43s/it]
 45%|████▌     | 4537/10000 [5:26:23<6:40:23,  4.40s/it]
 45%|████▌     | 4538/10000 [5:26:28<6:47:47,  4.48s/it]
 45%|████▌     | 4539/10000 [5:26:32<6:39:51,  4.39s/it]
 45%|████▌     | 4540/10000 [5:26:36<6:41:29,  4.41s/it]
                                                        

 45%|████▌     | 4540/10000 [5:26:36<6:41:29,  4.41s/it]
 45%|████▌     | 4541/10000 [5:26:41<6:45:12,  4.45s/it]
 45%|████▌     | 4542/10000 [5:26:45<6:36:38,  4.36s/it]
 45%|████▌     | 4543/10000 [5:26:49<6:38:50,  4.39s/it]
 45%|████▌     | 4544/10000 [5:26:54<6:41:14,  4.41s/it]
 45%|████▌     | 4545/10000 [5:26:58<6:36:50,  4.36s/it]
 45%|████▌     | 4546/10000 [5:27:03<6:40:09,  4.40s/it]
 45%|████▌     | 4547/10000 [5:27:07<6:36:16,  4.36s/it]
 45%|████▌     | 4548/10000 [5:27:11<6:38:39,  4.39s/it]
 45%|████▌     | 4549/10000 [5:27:16<6:42:13,  4.43s/it]
 46%|████▌     | 4550/10000 [5:27:20<6:44:18,  4.45s/it]
 46%|████▌     | 4551/10000 [5:27:25<6:36:08,  4.36s/it]
 46%|████▌     | 4552/10000 [5:27:29<6:38:36,  4.39s/it]
 46%|████▌     | 4553/10000 [5:27:33<6:40:17,  4.41s/it]
 46%|████▌     | 4554/10000 [5:27:38<6:33:49,  4.34s/it]
 46%|████▌     | 4555/10000 [5:27:42<6:35:59,  4.36s/it]
 46%|████▌     | 4556/10000 [5:27:47<6:38:36,  4.39s/it]
 46%|████▌     | 4557/10000 [5:27:51<6:32:55,  4.33s/it]
 46%|████▌     | 4558/10000 [5:27:55<6:36:11,  4.37s/it]
 46%|████▌     | 4559/10000 [5:28:00<6:40:17,  4.41s/it]
 46%|████▌     | 4560/10000 [5:28:04<6:42:24,  4.44s/it]
                                                        

 46%|████▌     | 4560/10000 [5:28:04<6:42:24,  4.44s/it]
 46%|████▌     | 4561/10000 [5:28:09<6:45:58,  4.48s/it]
 46%|████▌     | 4562/10000 [5:28:13<6:48:31,  4.51s/it]
 46%|████▌     | 4563/10000 [5:28:18<6:44:29,  4.46s/it]
 46%|████▌     | 4564/10000 [5:28:22<6:41:17,  4.43s/it]
 46%|████▌     | 4565/10000 [5:28:26<6:39:10,  4.41s/it]
 46%|████▌     | 4566/10000 [5:28:31<6:43:57,  4.46s/it]
 46%|████▌     | 4567/10000 [5:28:36<6:47:21,  4.50s/it]
 46%|████▌     | 4568/10000 [5:28:40<6:49:40,  4.53s/it]
 46%|████▌     | 4569/10000 [5:28:44<6:40:45,  4.43s/it]
 46%|████▌     | 4570/10000 [5:28:49<6:46:05,  4.49s/it]
 46%|████▌     | 4571/10000 [5:28:53<6:45:21,  4.48s/it]
 46%|████▌     | 4572/10000 [5:28:58<6:36:53,  4.39s/it]
 46%|████▌     | 4573/10000 [5:29:02<6:38:41,  4.41s/it]
 46%|████▌     | 4574/10000 [5:29:07<6:40:45,  4.43s/it]
 46%|████▌     | 4575/10000 [5:29:11<6:34:38,  4.36s/it]
 46%|████▌     | 4576/10000 [5:29:15<6:41:21,  4.44s/it]
 46%|████▌     | 4577/10000 [5:29:20<6:36:49,  4.39s/it]
 46%|████▌     | 4578/10000 [5:29:25<7:00:53,  4.66s/it]
 46%|████▌     | 4579/10000 [5:29:30<6:59:05,  4.64s/it]
 46%|████▌     | 4580/10000 [5:29:34<6:57:22,  4.62s/it]
                                                        

 46%|████▌     | 4580/10000 [5:29:34<6:57:22,  4.62s/it]
 46%|████▌     | 4581/10000 [5:29:38<6:48:09,  4.52s/it]
 46%|████▌     | 4582/10000 [5:29:43<6:55:06,  4.60s/it]
 46%|████▌     | 4583/10000 [5:29:47<6:44:14,  4.48s/it]
 46%|████▌     | 4584/10000 [5:29:52<6:42:10,  4.46s/it]
 46%|████▌     | 4585/10000 [5:29:57<6:53:13,  4.58s/it]
 46%|████▌     | 4586/10000 [5:30:01<6:49:16,  4.54s/it]
 46%|████▌     | 4587/10000 [5:30:05<6:38:51,  4.42s/it]
 46%|████▌     | 4588/10000 [5:30:10<6:39:23,  4.43s/it]
 46%|████▌     | 4589/10000 [5:30:14<6:39:35,  4.43s/it]
 46%|████▌     | 4590/10000 [5:30:18<6:32:10,  4.35s/it]
 46%|████▌     | 4591/10000 [5:30:23<6:34:37,  4.38s/it]
 46%|████▌     | 4592/10000 [5:30:27<6:40:52,  4.45s/it]
 46%|████▌     | 4593/10000 [5:30:31<6:32:42,  4.36s/it]
 46%|████▌     | 4594/10000 [5:30:36<6:35:27,  4.39s/it]
 46%|████▌     | 4595/10000 [5:30:40<6:35:20,  4.39s/it]
 46%|████▌     | 4596/10000 [5:30:45<6:36:50,  4.41s/it]
 46%|████▌     | 4597/10000 [5:30:49<6:37:01,  4.41s/it]
 46%|████▌     | 4598/10000 [5:30:54<6:40:22,  4.45s/it]
 46%|████▌     | 4599/10000 [5:30:58<6:33:34,  4.37s/it]
 46%|████▌     | 4600/10000 [5:31:03<6:39:31,  4.44s/it]
                                                        

 46%|████▌     | 4600/10000 [5:31:03<6:39:31,  4.44s/it]
 46%|████▌     | 4601/10000 [5:31:07<6:41:06,  4.46s/it]
 46%|████▌     | 4602/10000 [5:31:11<6:35:12,  4.39s/it]
 46%|████▌     | 4603/10000 [5:31:16<6:39:05,  4.44s/it]
 46%|████▌     | 4604/10000 [5:31:20<6:33:51,  4.38s/it]
 46%|████▌     | 4605/10000 [5:31:25<6:38:11,  4.43s/it]
 46%|████▌     | 4606/10000 [5:31:29<6:41:27,  4.47s/it]
 46%|████▌     | 4607/10000 [5:31:34<6:43:26,  4.49s/it]
 46%|████▌     | 4608/10000 [5:31:38<6:38:23,  4.43s/it]
 46%|████▌     | 4609/10000 [5:31:42<6:40:03,  4.45s/it]
 46%|████▌     | 4610/10000 [5:31:47<6:34:12,  4.39s/it]
 46%|████▌     | 4611/10000 [5:31:51<6:38:09,  4.43s/it]
 46%|████▌     | 4612/10000 [5:31:56<6:37:03,  4.42s/it]
 46%|████▌     | 4613/10000 [5:32:00<6:40:17,  4.46s/it]
 46%|████▌     | 4614/10000 [5:32:04<6:35:41,  4.41s/it]
 46%|████▌     | 4615/10000 [5:32:09<6:42:10,  4.48s/it]
 46%|████▌     | 4616/10000 [5:32:14<6:43:34,  4.50s/it]
 46%|████▌     | 4617/10000 [5:32:18<6:37:58,  4.44s/it]
 46%|████▌     | 4618/10000 [5:32:23<6:47:32,  4.54s/it]
 46%|████▌     | 4619/10000 [5:32:27<6:47:47,  4.55s/it]
 46%|████▌     | 4620/10000 [5:32:31<6:37:41,  4.44s/it]
                                                        

 46%|████▌     | 4620/10000 [5:32:31<6:37:41,  4.44s/it]
 46%|████▌     | 4621/10000 [5:32:36<6:39:34,  4.46s/it]
 46%|████▌     | 4622/10000 [5:32:40<6:39:52,  4.46s/it]
 46%|████▌     | 4623/10000 [5:32:45<6:32:16,  4.38s/it]
 46%|████▌     | 4624/10000 [5:32:49<6:37:51,  4.44s/it]
 46%|████▋     | 4625/10000 [5:32:54<6:36:08,  4.42s/it]
 46%|████▋     | 4626/10000 [5:32:58<6:28:37,  4.34s/it]
 46%|████▋     | 4627/10000 [5:33:02<6:31:04,  4.37s/it]
 46%|████▋     | 4628/10000 [5:33:07<6:32:29,  4.38s/it]
 46%|████▋     | 4629/10000 [5:33:11<6:27:07,  4.32s/it]
 46%|████▋     | 4630/10000 [5:33:15<6:30:00,  4.36s/it]
 46%|████▋     | 4631/10000 [5:33:20<6:32:18,  4.38s/it]
 46%|████▋     | 4632/10000 [5:33:24<6:27:14,  4.33s/it]
 46%|████▋     | 4633/10000 [5:33:28<6:29:53,  4.36s/it]
 46%|████▋     | 4634/10000 [5:33:33<6:32:22,  4.39s/it]
 46%|████▋     | 4635/10000 [5:33:37<6:26:29,  4.32s/it]
 46%|████▋     | 4636/10000 [5:33:41<6:30:12,  4.36s/it]
 46%|████▋     | 4637/10000 [5:33:46<6:38:17,  4.46s/it]
 46%|████▋     | 4638/10000 [5:33:50<6:29:50,  4.36s/it]
 46%|████▋     | 4639/10000 [5:33:55<6:31:16,  4.38s/it]
 46%|████▋     | 4640/10000 [5:33:59<6:25:12,  4.31s/it]
                                                        

 46%|████▋     | 4640/10000 [5:33:59<6:25:12,  4.31s/it]
 46%|████▋     | 4641/10000 [5:34:03<6:28:21,  4.35s/it]
 46%|████▋     | 4642/10000 [5:34:08<6:30:35,  4.37s/it]
 46%|████▋     | 4643/10000 [5:34:12<6:31:56,  4.39s/it]
 46%|████▋     | 4644/10000 [5:34:16<6:27:06,  4.34s/it]
 46%|████▋     | 4645/10000 [5:34:21<6:29:29,  4.36s/it]
 46%|████▋     | 4646/10000 [5:34:25<6:18:24,  4.24s/it]
 46%|████▋     | 4647/10000 [5:34:29<6:29:18,  4.36s/it]
 46%|████▋     | 4648/10000 [5:34:34<6:35:33,  4.43s/it]
 46%|████▋     | 4649/10000 [5:34:38<6:31:33,  4.39s/it]
 46%|████▋     | 4650/10000 [5:34:43<6:37:01,  4.45s/it]
 47%|████▋     | 4651/10000 [5:34:47<6:40:20,  4.49s/it]
 47%|████▋     | 4652/10000 [5:34:52<6:39:45,  4.49s/it]
 47%|████▋     | 4653/10000 [5:34:57<6:47:47,  4.58s/it]
 47%|████▋     | 4654/10000 [5:35:02<7:03:39,  4.75s/it]
 47%|████▋     | 4655/10000 [5:35:06<6:58:56,  4.70s/it]
 47%|████▋     | 4656/10000 [5:35:11<6:53:03,  4.64s/it]
 47%|████▋     | 4657/10000 [5:35:15<6:51:31,  4.62s/it]
 47%|████▋     | 4658/10000 [5:35:20<6:50:15,  4.61s/it]
 47%|████▋     | 4659/10000 [5:35:24<6:41:44,  4.51s/it]
 47%|████▋     | 4660/10000 [5:35:29<6:41:36,  4.51s/it]
                                                        

 47%|████▋     | 4660/10000 [5:35:29<6:41:36,  4.51s/it]
 47%|████▋     | 4661/10000 [5:35:33<6:43:28,  4.53s/it]
 47%|████▋     | 4662/10000 [5:35:38<6:36:59,  4.46s/it]
 47%|████▋     | 4663/10000 [5:35:43<6:48:12,  4.59s/it]
 47%|████▋     | 4664/10000 [5:35:47<6:52:42,  4.64s/it]
 47%|████▋     | 4665/10000 [5:35:52<6:43:00,  4.53s/it]
 47%|████▋     | 4666/10000 [5:35:56<6:43:16,  4.54s/it]
 47%|████▋     | 4667/10000 [5:36:01<6:44:13,  4.55s/it]
 47%|████▋     | 4668/10000 [5:36:05<6:37:02,  4.47s/it]
 47%|████▋     | 4669/10000 [5:36:10<6:40:03,  4.50s/it]
 47%|████▋     | 4670/10000 [5:36:14<6:41:08,  4.52s/it]
 47%|████▋     | 4671/10000 [5:36:18<6:35:12,  4.45s/it]
 47%|████▋     | 4672/10000 [5:36:23<6:42:20,  4.53s/it]
 47%|████▋     | 4673/10000 [5:36:27<6:35:34,  4.46s/it]
 47%|████▋     | 4674/10000 [5:36:32<6:38:27,  4.49s/it]
 47%|████▋     | 4675/10000 [5:36:37<6:40:53,  4.52s/it]
 47%|████▋     | 4676/10000 [5:36:42<7:02:21,  4.76s/it]
 47%|████▋     | 4677/10000 [5:36:46<6:47:30,  4.59s/it]
 47%|████▋     | 4678/10000 [5:36:51<6:47:08,  4.59s/it]
 47%|████▋     | 4679/10000 [5:36:55<6:46:51,  4.59s/it]
 47%|████▋     | 4680/10000 [5:36:59<6:34:54,  4.45s/it]
                                                        

 47%|████▋     | 4680/10000 [5:36:59<6:34:54,  4.45s/it]
 47%|████▋     | 4681/10000 [5:37:04<6:38:14,  4.49s/it]
 47%|████▋     | 4682/10000 [5:37:09<6:40:42,  4.52s/it]
 47%|████▋     | 4683/10000 [5:37:13<6:34:36,  4.45s/it]
 47%|████▋     | 4684/10000 [5:37:18<6:37:55,  4.49s/it]
 47%|████▋     | 4685/10000 [5:37:22<6:40:17,  4.52s/it]
 47%|████▋     | 4686/10000 [5:37:26<6:34:02,  4.45s/it]
 47%|████▋     | 4687/10000 [5:37:31<6:42:56,  4.55s/it]
 47%|████▋     | 4688/10000 [5:37:35<6:30:02,  4.41s/it]
 47%|████▋     | 4689/10000 [5:37:40<6:35:39,  4.47s/it]
 47%|████▋     | 4690/10000 [5:37:44<6:38:51,  4.51s/it]
 47%|████▋     | 4691/10000 [5:37:49<6:36:29,  4.48s/it]
 47%|████▋     | 4692/10000 [5:37:53<6:39:20,  4.51s/it]
 47%|████▋     | 4693/10000 [5:37:58<6:41:25,  4.54s/it]
 47%|████▋     | 4694/10000 [5:38:03<6:44:58,  4.58s/it]
 47%|████▋     | 4695/10000 [5:38:07<6:37:38,  4.50s/it]
 47%|████▋     | 4696/10000 [5:38:12<6:40:45,  4.53s/it]
 47%|████▋     | 4697/10000 [5:38:16<6:42:03,  4.55s/it]
 47%|████▋     | 4698/10000 [5:38:21<6:34:52,  4.47s/it]
 47%|████▋     | 4699/10000 [5:38:25<6:37:48,  4.50s/it]
 47%|████▋     | 4700/10000 [5:38:30<6:43:43,  4.57s/it]
                                                        
{'loss': '0.1019', 'grad_norm': '0.0002394', 'learning_rate': '2.741e-06', 'num_tokens': '1.342e+06', 'completions/mean_length': '4.737', 'completions/min_length': '4.5', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.737', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3142', 'rewards/reward_semantic/std': '0.03656', 'rewards/reward_length/mean': '-0.02312', 'rewards/reward_length/std': '0.006612', 'reward': '1.093', 'reward_std': '0.01086', 'frac_reward_zero_std': '0.75', 'kl': '2.975', 'entropy': '0.09086', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.483', 'epoch': '0.7415'}
{'loss': '0.1084', 'grad_norm': '0.0002898', 'learning_rate': '2.731e-06', 'num_tokens': '1.348e+06', 'completions/mean_length': '5.612', 'completions/min_length': '4.9', 'completions/max_length': '5.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.612', 'completions/min_terminated_length': '4.9', 'completions/max_terminated_length': '5.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3448', 'rewards/reward_semantic/std': '0.03149', 'rewards/reward_length/mean': '-0.03889', 'rewards/reward_length/std': '0.006688', 'reward': '1.101', 'reward_std': '0.009493', 'frac_reward_zero_std': '0.7', 'kl': '2.62', 'entropy': '0.1122', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.459', 'epoch': '0.7448'}
{'loss': '0.1031', 'grad_norm': '7.668', 'learning_rate': '2.721e-06', 'num_tokens': '1.354e+06', 'completions/mean_length': '4.912', 'completions/min_length': '4.8', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.912', 'completions/min_terminated_length': '4.8', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3527', 'rewards/reward_semantic/std': '0.05894', 'rewards/reward_length/mean': '-0.03269', 'rewards/reward_length/std': '0.003295', 'reward': '1.104', 'reward_std': '0.0178', 'frac_reward_zero_std': '0.75', 'kl': '2.728', 'entropy': '0.05996', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.375', 'epoch': '0.748'}
{'loss': '0.1152', 'grad_norm': '0.0002819', 'learning_rate': '2.711e-06', 'num_tokens': '1.36e+06', 'completions/mean_length': '4.938', 'completions/min_length': '4.7', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.938', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3484', 'rewards/reward_semantic/std': '0.05019', 'rewards/reward_length/mean': '-0.03599', 'rewards/reward_length/std': '0.003927', 'reward': '1.103', 'reward_std': '0.01502', 'frac_reward_zero_std': '0.8', 'kl': '2.727', 'entropy': '0.07169', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.477', 'epoch': '0.7513'}
{'loss': '0.1083', 'grad_norm': '0.02481', 'learning_rate': '2.701e-06', 'num_tokens': '1.366e+06', 'completions/mean_length': '4.981', 'completions/min_length': '4.55', 'completions/max_length': '5.4', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.981', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1513', 'rewards/reward_semantic/std': '0.06913', 'rewards/reward_length/mean': '-0.02954', 'rewards/reward_length/std': '0.007921', 'reward': '1.044', 'reward_std': '0.02069', 'frac_reward_zero_std': '0.7', 'kl': '3.074', 'entropy': '0.1243', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.401', 'epoch': '0.7546'}
{'loss': '0.109', 'grad_norm': '7.823', 'learning_rate': '2.691e-06', 'num_tokens': '1.372e+06', 'completions/mean_length': '4.594', 'completions/min_length': '4.25', 'completions/max_length': '4.9', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.594', 'completions/min_terminated_length': '4.25', 'completions/max_terminated_length': '4.9', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1979', 'rewards/reward_semantic/std': '0.02946', 'rewards/reward_length/mean': '-0.02898', 'rewards/reward_length/std': '0.003629', 'reward': '1.058', 'reward_std': '0.008981', 'frac_reward_zero_std': '0.75', 'kl': '2.939', 'entropy': '0.1584', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.429', 'epoch': '0.7579'}
{'loss': '0.1322', 'grad_norm': '0.002039', 'learning_rate': '2.681e-06', 'num_tokens': '1.378e+06', 'completions/mean_length': '4.787', 'completions/min_length': '4.6', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.787', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3344', 'rewards/reward_semantic/std': '0.02609', 'rewards/reward_length/mean': '-0.02554', 'rewards/reward_length/std': '0.005037', 'reward': '1.099', 'reward_std': '0.007904', 'frac_reward_zero_std': '0.8', 'kl': '2.978', 'entropy': '0.07512', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.346', 'epoch': '0.7612'}
{'loss': '0.124', 'grad_norm': '0.006539', 'learning_rate': '2.671e-06', 'num_tokens': '1.384e+06', 'completions/mean_length': '4.706', 'completions/min_length': '4.5', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.706', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.325', 'rewards/reward_semantic/std': '0.03745', 'rewards/reward_length/mean': '-0.02842', 'rewards/reward_length/std': '0.006621', 'reward': '1.096', 'reward_std': '0.01125', 'frac_reward_zero_std': '0.75', 'kl': '2.733', 'entropy': '0.09318', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.483', 'epoch': '0.7644'}
{'loss': '0.08677', 'grad_norm': '18.73', 'learning_rate': '2.661e-06', 'num_tokens': '1.39e+06', 'completions/mean_length': '5.119', 'completions/min_length': '4.95', 'completions/max_length': '5.45', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.119', 'completions/min_terminated_length': '4.95', 'completions/max_terminated_length': '5.45', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2837', 'rewards/reward_semantic/std': '0.03605', 'rewards/reward_length/mean': '-0.03506', 'rewards/reward_length/std': '0.005856', 'reward': '1.083', 'reward_std': '0.01068', 'frac_reward_zero_std': '0.75', 'kl': '2.676', 'entropy': '0.09411', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.511', 'epoch': '0.7677'}

 47%|████▋     | 4700/10000 [5:38:30<6:43:43,  4.57s/it]
 47%|████▋     | 4701/10000 [5:38:34<6:36:15,  4.49s/it]
 47%|████▋     | 4702/10000 [5:38:39<6:38:47,  4.52s/it]
 47%|████▋     | 4703/10000 [5:38:43<6:40:36,  4.54s/it]
 47%|████▋     | 4704/10000 [5:38:48<6:34:38,  4.47s/it]
 47%|████▋     | 4705/10000 [5:38:52<6:33:37,  4.46s/it]
 47%|████▋     | 4706/10000 [5:38:57<6:36:01,  4.49s/it]
 47%|████▋     | 4707/10000 [5:39:01<6:29:54,  4.42s/it]
 47%|████▋     | 4708/10000 [5:39:05<6:34:16,  4.47s/it]
 47%|████▋     | 4709/10000 [5:39:10<6:29:22,  4.42s/it]
 47%|████▋     | 4710/10000 [5:39:14<6:30:30,  4.43s/it]
 47%|████▋     | 4711/10000 [5:39:19<6:31:05,  4.44s/it]
 47%|████▋     | 4712/10000 [5:39:23<6:26:33,  4.39s/it]
 47%|████▋     | 4713/10000 [5:39:27<6:30:34,  4.43s/it]
 47%|████▋     | 4714/10000 [5:39:32<6:33:42,  4.47s/it]
 47%|████▋     | 4715/10000 [5:39:36<6:27:56,  4.40s/it]
 47%|████▋     | 4716/10000 [5:39:41<6:31:26,  4.44s/it]
 47%|████▋     | 4717/10000 [5:39:45<6:32:09,  4.45s/it]
 47%|████▋     | 4718/10000 [5:39:49<6:20:37,  4.32s/it]
 47%|████▋     | 4719/10000 [5:39:54<6:26:24,  4.39s/it]
 47%|████▋     | 4720/10000 [5:39:58<6:28:19,  4.41s/it]
                                                        

 47%|████▋     | 4720/10000 [5:39:58<6:28:19,  4.41s/it]
 47%|████▋     | 4721/10000 [5:40:03<6:25:37,  4.38s/it]
 47%|████▋     | 4722/10000 [5:40:07<6:31:01,  4.45s/it]
 47%|████▋     | 4723/10000 [5:40:12<6:34:51,  4.49s/it]
 47%|████▋     | 4724/10000 [5:40:16<6:31:43,  4.45s/it]
 47%|████▋     | 4725/10000 [5:40:21<6:35:14,  4.50s/it]
 47%|████▋     | 4726/10000 [5:40:25<6:37:21,  4.52s/it]
 47%|████▋     | 4727/10000 [5:40:30<6:31:14,  4.45s/it]
 47%|████▋     | 4728/10000 [5:40:34<6:34:24,  4.49s/it]
 47%|████▋     | 4729/10000 [5:40:39<6:36:23,  4.51s/it]
 47%|████▋     | 4730/10000 [5:40:43<6:29:45,  4.44s/it]
 47%|████▋     | 4731/10000 [5:40:48<6:53:32,  4.71s/it]
 47%|████▋     | 4732/10000 [5:40:53<6:48:54,  4.66s/it]
 47%|████▋     | 4733/10000 [5:40:57<6:38:10,  4.54s/it]
 47%|████▋     | 4734/10000 [5:41:02<6:38:17,  4.54s/it]
 47%|████▋     | 4735/10000 [5:41:06<6:38:23,  4.54s/it]
 47%|████▋     | 4736/10000 [5:41:11<6:36:20,  4.52s/it]
 47%|████▋     | 4737/10000 [5:41:15<6:29:26,  4.44s/it]
 47%|████▋     | 4738/10000 [5:41:20<6:31:52,  4.47s/it]
 47%|████▋     | 4739/10000 [5:41:24<6:33:38,  4.49s/it]
 47%|████▋     | 4740/10000 [5:41:28<6:31:15,  4.46s/it]
                                                        

 47%|████▋     | 4740/10000 [5:41:28<6:31:15,  4.46s/it]
 47%|████▋     | 4741/10000 [5:41:33<6:33:25,  4.49s/it]
 47%|████▋     | 4742/10000 [5:41:38<6:37:08,  4.53s/it]
 47%|████▋     | 4743/10000 [5:41:42<6:34:01,  4.50s/it]
 47%|████▋     | 4744/10000 [5:41:47<6:35:23,  4.51s/it]
 47%|████▋     | 4745/10000 [5:41:51<6:35:54,  4.52s/it]
 47%|████▋     | 4746/10000 [5:41:55<6:29:18,  4.45s/it]
 47%|████▋     | 4747/10000 [5:42:00<6:31:41,  4.47s/it]
 47%|████▋     | 4748/10000 [5:42:04<6:26:04,  4.41s/it]
 47%|████▋     | 4749/10000 [5:42:09<6:27:13,  4.42s/it]
 48%|████▊     | 4750/10000 [5:42:13<6:31:09,  4.47s/it]
 48%|████▊     | 4751/10000 [5:42:18<6:25:19,  4.40s/it]
 48%|████▊     | 4752/10000 [5:42:22<6:28:40,  4.44s/it]
 48%|████▊     | 4753/10000 [5:42:27<6:31:00,  4.47s/it]
 48%|████▊     | 4754/10000 [5:42:31<6:33:37,  4.50s/it]
 48%|████▊     | 4755/10000 [5:42:36<6:32:38,  4.49s/it]
 48%|████▊     | 4756/10000 [5:42:40<6:33:45,  4.51s/it]
 48%|████▊     | 4757/10000 [5:42:44<6:24:53,  4.40s/it]
 48%|████▊     | 4758/10000 [5:42:49<6:28:09,  4.44s/it]
 48%|████▊     | 4759/10000 [5:42:53<6:30:38,  4.47s/it]
 48%|████▊     | 4760/10000 [5:42:58<6:24:39,  4.40s/it]
                                                        

 48%|████▊     | 4760/10000 [5:42:58<6:24:39,  4.40s/it]
 48%|████▊     | 4761/10000 [5:43:02<6:28:17,  4.45s/it]
 48%|████▊     | 4762/10000 [5:43:07<6:30:43,  4.48s/it]
 48%|████▊     | 4763/10000 [5:43:11<6:24:47,  4.41s/it]
 48%|████▊     | 4764/10000 [5:43:16<6:28:28,  4.45s/it]
 48%|████▊     | 4765/10000 [5:43:20<6:31:01,  4.48s/it]
 48%|████▊     | 4766/10000 [5:43:24<6:24:37,  4.41s/it]
 48%|████▊     | 4767/10000 [5:43:29<6:41:54,  4.61s/it]
 48%|████▊     | 4768/10000 [5:43:34<6:39:53,  4.59s/it]
 48%|████▊     | 4769/10000 [5:43:38<6:38:28,  4.57s/it]
 48%|████▊     | 4770/10000 [5:43:42<6:24:02,  4.41s/it]
 48%|████▊     | 4771/10000 [5:43:47<6:35:10,  4.53s/it]
 48%|████▊     | 4772/10000 [5:43:52<6:38:12,  4.57s/it]
 48%|████▊     | 4773/10000 [5:43:57<6:43:12,  4.63s/it]
 48%|████▊     | 4774/10000 [5:44:01<6:41:31,  4.61s/it]
 48%|████▊     | 4775/10000 [5:44:06<6:38:52,  4.58s/it]
 48%|████▊     | 4776/10000 [5:44:10<6:30:15,  4.48s/it]
 48%|████▊     | 4777/10000 [5:44:15<6:32:54,  4.51s/it]
 48%|████▊     | 4778/10000 [5:44:19<6:33:29,  4.52s/it]
 48%|████▊     | 4779/10000 [5:44:23<6:26:57,  4.45s/it]
 48%|████▊     | 4780/10000 [5:44:28<6:29:13,  4.47s/it]
                                                        

 48%|████▊     | 4780/10000 [5:44:28<6:29:13,  4.47s/it]
 48%|████▊     | 4781/10000 [5:44:33<6:30:50,  4.49s/it]
 48%|████▊     | 4782/10000 [5:44:37<6:29:46,  4.48s/it]
 48%|████▊     | 4783/10000 [5:44:41<6:25:56,  4.44s/it]
 48%|████▊     | 4784/10000 [5:44:46<6:28:55,  4.47s/it]
 48%|████▊     | 4785/10000 [5:44:51<6:36:29,  4.56s/it]
 48%|████▊     | 4786/10000 [5:44:55<6:35:11,  4.55s/it]
 48%|████▊     | 4787/10000 [5:45:00<6:34:21,  4.54s/it]
 48%|████▊     | 4788/10000 [5:45:04<6:26:35,  4.45s/it]
 48%|████▊     | 4789/10000 [5:45:09<6:44:26,  4.66s/it]
 48%|████▊     | 4790/10000 [5:45:14<6:47:09,  4.69s/it]
 48%|████▊     | 4791/10000 [5:45:18<6:43:51,  4.65s/it]
 48%|████▊     | 4792/10000 [5:45:23<6:34:24,  4.54s/it]
 48%|████▊     | 4793/10000 [5:45:28<6:42:01,  4.63s/it]
 48%|████▊     | 4794/10000 [5:45:32<6:26:34,  4.46s/it]
 48%|████▊     | 4795/10000 [5:45:36<6:24:48,  4.44s/it]
 48%|████▊     | 4796/10000 [5:45:41<6:31:30,  4.51s/it]
 48%|████▊     | 4797/10000 [5:45:45<6:33:13,  4.53s/it]
 48%|████▊     | 4798/10000 [5:45:50<6:26:36,  4.46s/it]
 48%|████▊     | 4799/10000 [5:45:54<6:30:09,  4.50s/it]
 48%|████▊     | 4800/10000 [5:45:59<6:43:36,  4.66s/it]
                                                        

 48%|████▊     | 4800/10000 [5:45:59<6:43:36,  4.66s/it]
 48%|████▊     | 4801/10000 [5:46:03<6:33:36,  4.54s/it]
 48%|████▊     | 4802/10000 [5:46:08<6:34:41,  4.56s/it]
 48%|████▊     | 4803/10000 [5:46:12<6:27:25,  4.47s/it]
 48%|████▊     | 4804/10000 [5:46:17<6:30:10,  4.51s/it]
 48%|████▊     | 4805/10000 [5:46:21<6:32:02,  4.53s/it]
 48%|████▊     | 4806/10000 [5:46:26<6:33:17,  4.54s/it]
 48%|████▊     | 4807/10000 [5:46:30<6:23:46,  4.43s/it]
 48%|████▊     | 4808/10000 [5:46:35<6:32:23,  4.53s/it]
 48%|████▊     | 4809/10000 [5:46:40<6:35:34,  4.57s/it]
 48%|████▊     | 4810/10000 [5:46:44<6:30:28,  4.51s/it]
 48%|████▊     | 4811/10000 [5:46:49<6:35:02,  4.57s/it]
 48%|████▊     | 4812/10000 [5:46:54<6:40:36,  4.63s/it]
 48%|████▊     | 4813/10000 [5:46:58<6:34:43,  4.57s/it]
 48%|████▊     | 4814/10000 [5:47:02<6:32:31,  4.54s/it]
 48%|████▊     | 4815/10000 [5:47:07<6:33:05,  4.55s/it]
 48%|████▊     | 4816/10000 [5:47:11<6:26:20,  4.47s/it]
 48%|████▊     | 4817/10000 [5:47:16<6:32:34,  4.54s/it]
 48%|████▊     | 4818/10000 [5:47:21<6:33:50,  4.56s/it]
 48%|████▊     | 4819/10000 [5:47:25<6:26:44,  4.48s/it]
 48%|████▊     | 4820/10000 [5:47:29<6:29:10,  4.51s/it]
                                                        

 48%|████▊     | 4820/10000 [5:47:29<6:29:10,  4.51s/it]
 48%|████▊     | 4821/10000 [5:47:34<6:24:46,  4.46s/it]
 48%|████▊     | 4822/10000 [5:47:38<6:14:43,  4.34s/it]
 48%|████▊     | 4823/10000 [5:47:43<6:30:13,  4.52s/it]
 48%|████▊     | 4824/10000 [5:47:47<6:28:53,  4.51s/it]
 48%|████▊     | 4825/10000 [5:47:52<6:25:28,  4.47s/it]
 48%|████▊     | 4826/10000 [5:47:56<6:28:42,  4.51s/it]
 48%|████▊     | 4827/10000 [5:48:01<6:27:26,  4.49s/it]
 48%|████▊     | 4828/10000 [5:48:05<6:30:27,  4.53s/it]
 48%|████▊     | 4829/10000 [5:48:10<6:22:28,  4.44s/it]
 48%|████▊     | 4830/10000 [5:48:14<6:26:47,  4.49s/it]
 48%|████▊     | 4831/10000 [5:48:19<6:26:33,  4.49s/it]
 48%|████▊     | 4832/10000 [5:48:23<6:19:06,  4.40s/it]
 48%|████▊     | 4833/10000 [5:48:27<6:21:05,  4.43s/it]
 48%|████▊     | 4834/10000 [5:48:32<6:32:19,  4.56s/it]
 48%|████▊     | 4835/10000 [5:48:37<6:43:35,  4.69s/it]
 48%|████▊     | 4836/10000 [5:48:41<6:33:30,  4.57s/it]
 48%|████▊     | 4837/10000 [5:48:46<6:34:06,  4.58s/it]
 48%|████▊     | 4838/10000 [5:48:51<6:34:29,  4.59s/it]
 48%|████▊     | 4839/10000 [5:48:55<6:30:21,  4.54s/it]
 48%|████▊     | 4840/10000 [5:49:00<6:29:13,  4.53s/it]
                                                        

 48%|████▊     | 4840/10000 [5:49:00<6:29:13,  4.53s/it]
 48%|████▊     | 4841/10000 [5:49:04<6:30:32,  4.54s/it]
 48%|████▊     | 4842/10000 [5:49:09<6:27:33,  4.51s/it]
 48%|████▊     | 4843/10000 [5:49:13<6:21:11,  4.43s/it]
 48%|████▊     | 4844/10000 [5:49:17<6:24:57,  4.48s/it]
 48%|████▊     | 4845/10000 [5:49:22<6:26:51,  4.50s/it]
 48%|████▊     | 4846/10000 [5:49:26<6:21:16,  4.44s/it]
 48%|████▊     | 4847/10000 [5:49:31<6:24:50,  4.48s/it]
 48%|████▊     | 4848/10000 [5:49:35<6:26:22,  4.50s/it]
 48%|████▊     | 4849/10000 [5:49:40<6:29:56,  4.54s/it]
 48%|████▊     | 4850/10000 [5:49:44<6:24:01,  4.47s/it]
 49%|████▊     | 4851/10000 [5:49:49<6:26:46,  4.51s/it]
 49%|████▊     | 4852/10000 [5:49:54<6:29:30,  4.54s/it]
 49%|████▊     | 4853/10000 [5:49:58<6:21:37,  4.45s/it]
 49%|████▊     | 4854/10000 [5:50:02<6:25:56,  4.50s/it]
 49%|████▊     | 4855/10000 [5:50:07<6:28:32,  4.53s/it]
 49%|████▊     | 4856/10000 [5:50:12<6:40:17,  4.67s/it]
 49%|████▊     | 4857/10000 [5:50:16<6:32:32,  4.58s/it]
 49%|████▊     | 4858/10000 [5:50:21<6:33:41,  4.59s/it]
 49%|████▊     | 4859/10000 [5:50:26<6:35:39,  4.62s/it]
 49%|████▊     | 4860/10000 [5:50:30<6:35:44,  4.62s/it]
                                                        

 49%|████▊     | 4860/10000 [5:50:30<6:35:44,  4.62s/it]
 49%|████▊     | 4861/10000 [5:50:35<6:28:01,  4.53s/it]
 49%|████▊     | 4862/10000 [5:50:39<6:30:39,  4.56s/it]
 49%|████▊     | 4863/10000 [5:50:44<6:25:58,  4.51s/it]
 49%|████▊     | 4864/10000 [5:50:48<6:21:06,  4.45s/it]
 49%|████▊     | 4865/10000 [5:50:53<6:25:09,  4.50s/it]
 49%|████▊     | 4866/10000 [5:50:57<6:28:18,  4.54s/it]
 49%|████▊     | 4867/10000 [5:51:02<6:22:20,  4.47s/it]
 49%|████▊     | 4868/10000 [5:51:06<6:24:36,  4.50s/it]
 49%|████▊     | 4869/10000 [5:51:11<6:26:58,  4.53s/it]
 49%|████▊     | 4870/10000 [5:51:15<6:31:40,  4.58s/it]
 49%|████▊     | 4871/10000 [5:51:20<6:24:21,  4.50s/it]
 49%|████▊     | 4872/10000 [5:51:24<6:26:51,  4.53s/it]
 49%|████▊     | 4873/10000 [5:51:29<6:27:56,  4.54s/it]
 49%|████▊     | 4874/10000 [5:51:33<6:29:12,  4.56s/it]
 49%|████▉     | 4875/10000 [5:51:38<6:24:03,  4.50s/it]
 49%|████▉     | 4876/10000 [5:51:42<6:26:12,  4.52s/it]
 49%|████▉     | 4877/10000 [5:51:47<6:27:50,  4.54s/it]
 49%|████▉     | 4878/10000 [5:51:51<6:22:54,  4.49s/it]
 49%|████▉     | 4879/10000 [5:51:56<6:18:02,  4.43s/it]
 49%|████▉     | 4880/10000 [5:52:00<6:21:56,  4.48s/it]
                                                        
{'loss': '0.09924', 'grad_norm': '15.8', 'learning_rate': '2.651e-06', 'num_tokens': '1.396e+06', 'completions/mean_length': '4.688', 'completions/min_length': '4.45', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.688', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1565', 'rewards/reward_semantic/std': '0.05735', 'rewards/reward_length/mean': '-0.0249', 'rewards/reward_length/std': '0.005079', 'reward': '1.046', 'reward_std': '0.01704', 'frac_reward_zero_std': '0.7', 'kl': '3.066', 'entropy': '0.0909', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.498', 'epoch': '0.771'}
{'loss': '0.124', 'grad_norm': '0.00011', 'learning_rate': '2.641e-06', 'num_tokens': '1.402e+06', 'completions/mean_length': '4.638', 'completions/min_length': '4.55', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.638', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3507', 'rewards/reward_semantic/std': '0.01521', 'rewards/reward_length/mean': '-0.03311', 'rewards/reward_length/std': '0.001381', 'reward': '1.104', 'reward_std': '0.004494', 'frac_reward_zero_std': '0.95', 'kl': '3.066', 'entropy': '0.05209', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.404', 'epoch': '0.7743'}
{'loss': '0.1273', 'grad_norm': '0.004095', 'learning_rate': '2.631e-06', 'num_tokens': '1.408e+06', 'completions/mean_length': '5.237', 'completions/min_length': '5', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.237', 'completions/min_terminated_length': '5', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2024', 'rewards/reward_semantic/std': '0.0193', 'rewards/reward_length/mean': '-0.03127', 'rewards/reward_length/std': '0.004143', 'reward': '1.059', 'reward_std': '0.005906', 'frac_reward_zero_std': '0.8', 'kl': '2.674', 'entropy': '0.07824', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.488', 'epoch': '0.7776'}
{'loss': '0.1162', 'grad_norm': '0.001899', 'learning_rate': '2.621e-06', 'num_tokens': '1.414e+06', 'completions/mean_length': '4.775', 'completions/min_length': '4.6', 'completions/max_length': '5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.775', 'completions/min_terminated_length': '4.6', 'completions/max_terminated_length': '5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3797', 'rewards/reward_semantic/std': '0.04423', 'rewards/reward_length/mean': '-0.03721', 'rewards/reward_length/std': '0.004969', 'reward': '1.112', 'reward_std': '0.01316', 'frac_reward_zero_std': '0.75', 'kl': '3.106', 'entropy': '0.134', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.439', 'epoch': '0.7808'}
{'loss': '0.1561', 'grad_norm': '11.22', 'learning_rate': '2.611e-06', 'num_tokens': '1.421e+06', 'completions/mean_length': '5.037', 'completions/min_length': '4.55', 'completions/max_length': '5.5', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.037', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.5', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3', 'rewards/reward_semantic/std': '0.07554', 'rewards/reward_length/mean': '-0.04067', 'rewards/reward_length/std': '0.007987', 'reward': '1.088', 'reward_std': '0.02277', 'frac_reward_zero_std': '0.65', 'kl': '2.858', 'entropy': '0.1415', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.499', 'epoch': '0.7841'}
{'loss': '0.1324', 'grad_norm': '11.37', 'learning_rate': '2.601e-06', 'num_tokens': '1.427e+06', 'completions/mean_length': '4.775', 'completions/min_length': '4.15', 'completions/max_length': '5.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.775', 'completions/min_terminated_length': '4.15', 'completions/max_terminated_length': '5.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4747', 'rewards/reward_semantic/std': '0.04171', 'rewards/reward_length/mean': '-0.04172', 'rewards/reward_length/std': '0.009449', 'reward': '1.14', 'reward_std': '0.01264', 'frac_reward_zero_std': '0.6', 'kl': '2.748', 'entropy': '0.1442', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.537', 'epoch': '0.7874'}
{'loss': '0.1187', 'grad_norm': '0.0007952', 'learning_rate': '2.591e-06', 'num_tokens': '1.433e+06', 'completions/mean_length': '4.844', 'completions/min_length': '4.65', 'completions/max_length': '5.15', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.844', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.15', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3752', 'rewards/reward_semantic/std': '0.06306', 'rewards/reward_length/mean': '-0.0531', 'rewards/reward_length/std': '0.008367', 'reward': '1.11', 'reward_std': '0.01898', 'frac_reward_zero_std': '0.55', 'kl': '2.604', 'entropy': '0.1678', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.493', 'epoch': '0.7907'}
{'loss': '0.1093', 'grad_norm': '0.02361', 'learning_rate': '2.581e-06', 'num_tokens': '1.439e+06', 'completions/mean_length': '4.513', 'completions/min_length': '3.95', 'completions/max_length': '5.25', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.513', 'completions/min_terminated_length': '3.95', 'completions/max_terminated_length': '5.25', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3861', 'rewards/reward_semantic/std': '0.06718', 'rewards/reward_length/mean': '-0.03801', 'rewards/reward_length/std': '0.008073', 'reward': '1.114', 'reward_std': '0.02019', 'frac_reward_zero_std': '0.5', 'kl': '2.914', 'entropy': '0.2029', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.489', 'epoch': '0.794'}
{'loss': '0.1311', 'grad_norm': '0.002091', 'learning_rate': '2.571e-06', 'num_tokens': '1.444e+06', 'completions/mean_length': '4.631', 'completions/min_length': '4.35', 'completions/max_length': '4.95', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.631', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.95', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.5013', 'rewards/reward_semantic/std': '0.03104', 'rewards/reward_length/mean': '-0.0353', 'rewards/reward_length/std': '0.003799', 'reward': '1.149', 'reward_std': '0.009502', 'frac_reward_zero_std': '0.75', 'kl': '2.677', 'entropy': '0.1024', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.516', 'epoch': '0.7972'}

 49%|████▉     | 4880/10000 [5:52:00<6:21:56,  4.48s/it]
 49%|████▉     | 4881/10000 [5:52:05<6:25:37,  4.52s/it]
 49%|████▉     | 4882/10000 [5:52:09<6:20:21,  4.46s/it]
 49%|████▉     | 4883/10000 [5:52:14<6:24:46,  4.51s/it]
 49%|████▉     | 4884/10000 [5:52:18<6:29:36,  4.57s/it]
 49%|████▉     | 4885/10000 [5:52:23<6:30:36,  4.58s/it]
 49%|████▉     | 4886/10000 [5:52:27<6:23:00,  4.49s/it]
 49%|████▉     | 4887/10000 [5:52:32<6:25:54,  4.53s/it]
 49%|████▉     | 4888/10000 [5:52:37<6:28:04,  4.55s/it]
 49%|████▉     | 4889/10000 [5:52:41<6:21:44,  4.48s/it]
 49%|████▉     | 4890/10000 [5:52:46<6:24:13,  4.51s/it]
 49%|████▉     | 4891/10000 [5:52:50<6:25:35,  4.53s/it]
 49%|████▉     | 4892/10000 [5:52:55<6:27:00,  4.55s/it]
 49%|████▉     | 4893/10000 [5:52:59<6:21:01,  4.48s/it]
 49%|████▉     | 4894/10000 [5:53:03<6:17:56,  4.44s/it]
 49%|████▉     | 4895/10000 [5:53:08<6:21:27,  4.48s/it]
 49%|████▉     | 4896/10000 [5:53:13<6:34:40,  4.64s/it]
 49%|████▉     | 4897/10000 [5:53:17<6:25:29,  4.53s/it]
 49%|████▉     | 4898/10000 [5:53:22<6:26:57,  4.55s/it]
 49%|████▉     | 4899/10000 [5:53:26<6:29:19,  4.58s/it]
 49%|████▉     | 4900/10000 [5:53:31<6:21:38,  4.49s/it]
                                                        

 49%|████▉     | 4900/10000 [5:53:31<6:21:38,  4.49s/it]
 49%|████▉     | 4901/10000 [5:53:35<6:20:54,  4.48s/it]
 49%|████▉     | 4902/10000 [5:53:40<6:24:36,  4.53s/it]
 49%|████▉     | 4903/10000 [5:53:44<6:19:40,  4.47s/it]
 49%|████▉     | 4904/10000 [5:53:48<6:15:03,  4.42s/it]
 49%|████▉     | 4905/10000 [5:53:53<6:18:17,  4.45s/it]
 49%|████▉     | 4906/10000 [5:53:58<6:21:27,  4.49s/it]
 49%|████▉     | 4907/10000 [5:54:02<6:15:25,  4.42s/it]
 49%|████▉     | 4908/10000 [5:54:06<6:12:24,  4.39s/it]
 49%|████▉     | 4909/10000 [5:54:11<6:15:33,  4.43s/it]
 49%|████▉     | 4910/10000 [5:54:15<6:18:38,  4.46s/it]
 49%|████▉     | 4911/10000 [5:54:19<6:13:54,  4.41s/it]
 49%|████▉     | 4912/10000 [5:54:24<6:18:14,  4.46s/it]
 49%|████▉     | 4913/10000 [5:54:29<6:20:47,  4.49s/it]
 49%|████▉     | 4914/10000 [5:54:33<6:15:23,  4.43s/it]
 49%|████▉     | 4915/10000 [5:54:38<6:19:32,  4.48s/it]
 49%|████▉     | 4916/10000 [5:54:42<6:22:16,  4.51s/it]
 49%|████▉     | 4917/10000 [5:54:47<6:24:22,  4.54s/it]
 49%|████▉     | 4918/10000 [5:54:51<6:18:13,  4.47s/it]
 49%|████▉     | 4919/10000 [5:54:56<6:21:12,  4.50s/it]
 49%|████▉     | 4920/10000 [5:55:00<6:23:30,  4.53s/it]
                                                        

 49%|████▉     | 4920/10000 [5:55:00<6:23:30,  4.53s/it]
 49%|████▉     | 4921/10000 [5:55:05<6:24:49,  4.55s/it]
 49%|████▉     | 4922/10000 [5:55:09<6:18:43,  4.47s/it]
 49%|████▉     | 4923/10000 [5:55:14<6:23:42,  4.53s/it]
 49%|████▉     | 4924/10000 [5:55:18<6:24:43,  4.55s/it]
 49%|████▉     | 4925/10000 [5:55:23<6:21:59,  4.52s/it]
 49%|████▉     | 4926/10000 [5:55:27<6:23:20,  4.53s/it]
 49%|████▉     | 4927/10000 [5:55:32<6:22:53,  4.53s/it]
 49%|████▉     | 4928/10000 [5:55:36<6:24:51,  4.55s/it]
 49%|████▉     | 4929/10000 [5:55:41<6:19:21,  4.49s/it]
 49%|████▉     | 4930/10000 [5:55:45<6:21:28,  4.51s/it]
 49%|████▉     | 4931/10000 [5:55:50<6:23:51,  4.54s/it]
 49%|████▉     | 4932/10000 [5:55:55<6:38:06,  4.71s/it]
 49%|████▉     | 4933/10000 [5:55:59<6:27:54,  4.59s/it]
 49%|████▉     | 4934/10000 [5:56:04<6:28:30,  4.60s/it]
 49%|████▉     | 4935/10000 [5:56:09<6:31:00,  4.63s/it]
 49%|████▉     | 4936/10000 [5:56:13<6:22:36,  4.53s/it]
 49%|████▉     | 4937/10000 [5:56:18<6:24:04,  4.55s/it]
 49%|████▉     | 4938/10000 [5:56:22<6:25:34,  4.57s/it]
 49%|████▉     | 4939/10000 [5:56:27<6:27:34,  4.59s/it]
 49%|████▉     | 4940/10000 [5:56:31<6:20:06,  4.51s/it]
                                                        

 49%|████▉     | 4940/10000 [5:56:31<6:20:06,  4.51s/it]
 49%|████▉     | 4941/10000 [5:56:36<6:21:45,  4.53s/it]
 49%|████▉     | 4942/10000 [5:56:40<6:24:09,  4.56s/it]
 49%|████▉     | 4943/10000 [5:56:45<6:24:52,  4.57s/it]
 49%|████▉     | 4944/10000 [5:56:49<6:17:35,  4.48s/it]
 49%|████▉     | 4945/10000 [5:56:54<6:23:15,  4.55s/it]
 49%|████▉     | 4946/10000 [5:56:59<6:24:10,  4.56s/it]
 49%|████▉     | 4947/10000 [5:57:03<6:17:27,  4.48s/it]
 49%|████▉     | 4948/10000 [5:57:07<6:20:45,  4.52s/it]
 49%|████▉     | 4949/10000 [5:57:12<6:22:34,  4.54s/it]
 50%|████▉     | 4950/10000 [5:57:17<6:23:13,  4.55s/it]
 50%|████▉     | 4951/10000 [5:57:21<6:10:48,  4.41s/it]
 50%|████▉     | 4952/10000 [5:57:25<6:15:48,  4.47s/it]
 50%|████▉     | 4953/10000 [5:57:30<6:18:52,  4.50s/it]
 50%|████▉     | 4954/10000 [5:57:34<6:13:29,  4.44s/it]
 50%|████▉     | 4955/10000 [5:57:39<6:16:37,  4.48s/it]
 50%|████▉     | 4956/10000 [5:57:43<6:18:48,  4.51s/it]
 50%|████▉     | 4957/10000 [5:57:48<6:20:13,  4.52s/it]
 50%|████▉     | 4958/10000 [5:57:52<6:14:19,  4.45s/it]
 50%|████▉     | 4959/10000 [5:57:57<6:18:10,  4.50s/it]
 50%|████▉     | 4960/10000 [5:58:01<6:20:20,  4.53s/it]
                                                        

 50%|████▉     | 4960/10000 [5:58:01<6:20:20,  4.53s/it]
 50%|████▉     | 4961/10000 [5:58:06<6:25:09,  4.59s/it]
 50%|████▉     | 4962/10000 [5:58:10<6:17:00,  4.49s/it]
 50%|████▉     | 4963/10000 [5:58:15<6:18:51,  4.51s/it]
 50%|████▉     | 4964/10000 [5:58:19<6:19:26,  4.52s/it]
 50%|████▉     | 4965/10000 [5:58:24<6:13:02,  4.45s/it]
 50%|████▉     | 4966/10000 [5:58:29<6:21:26,  4.55s/it]
 50%|████▉     | 4967/10000 [5:58:33<6:24:14,  4.58s/it]
 50%|████▉     | 4968/10000 [5:58:38<6:22:45,  4.56s/it]
 50%|████▉     | 4969/10000 [5:58:42<6:15:19,  4.48s/it]
 50%|████▉     | 4970/10000 [5:58:47<6:17:12,  4.50s/it]
 50%|████▉     | 4971/10000 [5:58:52<6:38:02,  4.75s/it]
 50%|████▉     | 4972/10000 [5:58:56<6:27:17,  4.62s/it]
 50%|████▉     | 4973/10000 [5:59:00<6:16:50,  4.50s/it]
 50%|████▉     | 4974/10000 [5:59:05<6:18:57,  4.52s/it]
 50%|████▉     | 4975/10000 [5:59:10<6:20:48,  4.55s/it]
 50%|████▉     | 4976/10000 [5:59:14<6:15:01,  4.48s/it]
 50%|████▉     | 4977/10000 [5:59:19<6:19:17,  4.53s/it]
 50%|████▉     | 4978/10000 [5:59:23<6:20:40,  4.55s/it]
 50%|████▉     | 4979/10000 [5:59:28<6:22:18,  4.57s/it]
 50%|████▉     | 4980/10000 [5:59:32<6:14:49,  4.48s/it]
                                                        

 50%|████▉     | 4980/10000 [5:59:32<6:14:49,  4.48s/it]
 50%|████▉     | 4981/10000 [5:59:37<6:24:37,  4.60s/it]
 50%|████▉     | 4982/10000 [5:59:42<6:24:17,  4.60s/it]
 50%|████▉     | 4983/10000 [5:59:46<6:30:37,  4.67s/it]
 50%|████▉     | 4984/10000 [5:59:51<6:21:21,  4.56s/it]
 50%|████▉     | 4985/10000 [5:59:56<6:33:32,  4.71s/it]
 50%|████▉     | 4986/10000 [6:00:00<6:29:57,  4.67s/it]
 50%|████▉     | 4987/10000 [6:00:05<6:25:19,  4.61s/it]
 50%|████▉     | 4988/10000 [6:00:09<6:16:18,  4.50s/it]
 50%|████▉     | 4989/10000 [6:00:14<6:15:52,  4.50s/it]
 50%|████▉     | 4990/10000 [6:00:18<6:17:13,  4.52s/it]
 50%|████▉     | 4991/10000 [6:00:23<6:18:38,  4.54s/it]
 50%|████▉     | 4992/10000 [6:00:27<6:11:32,  4.45s/it]
 50%|████▉     | 4993/10000 [6:00:31<6:14:03,  4.48s/it]
 50%|████▉     | 4994/10000 [6:00:36<6:15:53,  4.51s/it]
 50%|████▉     | 4995/10000 [6:00:40<6:09:49,  4.43s/it]
 50%|████▉     | 4996/10000 [6:00:45<6:13:50,  4.48s/it]
 50%|████▉     | 4997/10000 [6:00:49<6:16:39,  4.52s/it]
 50%|████▉     | 4998/10000 [6:00:54<6:20:50,  4.57s/it]
 50%|████▉     | 4999/10000 [6:00:58<6:13:47,  4.48s/it]
 50%|█████     | 5000/10000 [6:01:03<6:16:31,  4.52s/it]
                                                        

 50%|█████     | 5000/10000 [6:01:03<6:16:31,  4.52s/it]
 50%|█████     | 5001/10000 [6:01:09<6:46:18,  4.88s/it]
 50%|█████     | 5002/10000 [6:01:13<6:36:41,  4.76s/it]
 50%|█████     | 5003/10000 [6:01:18<6:31:48,  4.70s/it]
 50%|█████     | 5004/10000 [6:01:22<6:28:37,  4.67s/it]
 50%|█████     | 5005/10000 [6:01:27<6:26:26,  4.64s/it]
 50%|█████     | 5006/10000 [6:01:31<6:18:05,  4.54s/it]
 50%|█████     | 5007/10000 [6:01:36<6:20:47,  4.58s/it]
 50%|█████     | 5008/10000 [6:01:41<6:21:02,  4.58s/it]
 50%|█████     | 5009/10000 [6:01:45<6:26:07,  4.64s/it]
 50%|█████     | 5010/10000 [6:01:50<6:17:36,  4.54s/it]
 50%|█████     | 5011/10000 [6:01:55<6:26:00,  4.64s/it]
 50%|█████     | 5012/10000 [6:01:59<6:24:41,  4.63s/it]
 50%|█████     | 5013/10000 [6:02:04<6:23:04,  4.61s/it]
 50%|█████     | 5014/10000 [6:02:08<6:15:16,  4.52s/it]
 50%|█████     | 5015/10000 [6:02:13<6:17:01,  4.54s/it]
 50%|█████     | 5016/10000 [6:02:17<6:18:34,  4.56s/it]
 50%|█████     | 5017/10000 [6:02:22<6:14:20,  4.51s/it]
 50%|█████     | 5018/10000 [6:02:26<6:16:31,  4.53s/it]
 50%|█████     | 5019/10000 [6:02:31<6:17:45,  4.55s/it]
 50%|█████     | 5020/10000 [6:02:35<6:18:30,  4.56s/it]
                                                        

 50%|█████     | 5020/10000 [6:02:35<6:18:30,  4.56s/it]
 50%|█████     | 5021/10000 [6:02:40<6:11:46,  4.48s/it]
 50%|█████     | 5022/10000 [6:02:44<6:13:30,  4.50s/it]
 50%|█████     | 5023/10000 [6:02:49<6:14:46,  4.52s/it]
 50%|█████     | 5024/10000 [6:02:53<6:09:07,  4.45s/it]
 50%|█████     | 5025/10000 [6:02:58<6:12:11,  4.49s/it]
 50%|█████     | 5026/10000 [6:03:02<6:14:18,  4.52s/it]
 50%|█████     | 5027/10000 [6:03:07<6:14:21,  4.52s/it]
 50%|█████     | 5028/10000 [6:03:12<6:27:24,  4.68s/it]
 50%|█████     | 5029/10000 [6:03:16<6:28:38,  4.69s/it]
 50%|█████     | 5030/10000 [6:03:21<6:25:11,  4.65s/it]
 50%|█████     | 5031/10000 [6:03:26<6:23:20,  4.63s/it]
 50%|█████     | 5032/10000 [6:03:30<6:21:20,  4.61s/it]
 50%|█████     | 5033/10000 [6:03:34<6:12:30,  4.50s/it]
 50%|█████     | 5034/10000 [6:03:39<6:13:36,  4.51s/it]
 50%|█████     | 5035/10000 [6:03:44<6:16:47,  4.55s/it]
 50%|█████     | 5036/10000 [6:03:48<6:09:35,  4.47s/it]
 50%|█████     | 5037/10000 [6:03:53<6:17:06,  4.56s/it]
 50%|█████     | 5038/10000 [6:03:57<6:17:04,  4.56s/it]
 50%|█████     | 5039/10000 [6:04:02<6:18:33,  4.58s/it]
 50%|█████     | 5040/10000 [6:04:06<6:10:46,  4.49s/it]
                                                        

 50%|█████     | 5040/10000 [6:04:06<6:10:46,  4.49s/it]
 50%|█████     | 5041/10000 [6:04:11<6:12:19,  4.50s/it]
 50%|█████     | 5042/10000 [6:04:15<6:13:24,  4.52s/it]
 50%|█████     | 5043/10000 [6:04:20<6:15:05,  4.54s/it]
 50%|█████     | 5044/10000 [6:04:24<6:07:50,  4.45s/it]
 50%|█████     | 5045/10000 [6:04:29<6:07:49,  4.45s/it]
 50%|█████     | 5046/10000 [6:04:33<6:08:07,  4.46s/it]
 50%|█████     | 5047/10000 [6:04:37<6:00:21,  4.37s/it]
 50%|█████     | 5048/10000 [6:04:42<6:02:11,  4.39s/it]
 50%|█████     | 5049/10000 [6:04:46<6:04:22,  4.42s/it]
 50%|█████     | 5050/10000 [6:04:50<6:05:05,  4.43s/it]
 51%|█████     | 5051/10000 [6:04:55<5:58:23,  4.35s/it]
 51%|█████     | 5052/10000 [6:04:59<5:58:52,  4.35s/it]
 51%|█████     | 5053/10000 [6:05:03<6:01:16,  4.38s/it]
 51%|█████     | 5054/10000 [6:05:08<5:58:09,  4.34s/it]
 51%|█████     | 5055/10000 [6:05:12<6:00:22,  4.37s/it]
 51%|█████     | 5056/10000 [6:05:17<6:12:08,  4.52s/it]
 51%|█████     | 5057/10000 [6:05:21<6:10:38,  4.50s/it]
 51%|█████     | 5058/10000 [6:05:25<5:56:22,  4.33s/it]
 51%|█████     | 5059/10000 [6:05:30<5:59:15,  4.36s/it]
 51%|█████     | 5060/10000 [6:05:34<6:01:37,  4.39s/it]
                                                        
{'loss': '0.13', 'grad_norm': '0.01318', 'learning_rate': '2.561e-06', 'num_tokens': '1.45e+06', 'completions/mean_length': '4.556', 'completions/min_length': '4.4', 'completions/max_length': '4.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.556', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '4.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3425', 'rewards/reward_semantic/std': '0.01389', 'rewards/reward_length/mean': '-0.02761', 'rewards/reward_length/std': '0.002885', 'reward': '1.101', 'reward_std': '0.00431', 'frac_reward_zero_std': '0.9', 'kl': '2.781', 'entropy': '0.08949', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.475', 'epoch': '0.8005'}
{'loss': '0.1168', 'grad_norm': '0.000374', 'learning_rate': '2.551e-06', 'num_tokens': '1.457e+06', 'completions/mean_length': '4.838', 'completions/min_length': '4.5', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.838', 'completions/min_terminated_length': '4.5', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2471', 'rewards/reward_semantic/std': '0.04294', 'rewards/reward_length/mean': '-0.02563', 'rewards/reward_length/std': '0.005615', 'reward': '1.073', 'reward_std': '0.01276', 'frac_reward_zero_std': '0.75', 'kl': '2.836', 'entropy': '0.08272', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.506', 'epoch': '0.8038'}
{'loss': '0.1053', 'grad_norm': '0.08648', 'learning_rate': '2.541e-06', 'num_tokens': '1.462e+06', 'completions/mean_length': '4.588', 'completions/min_length': '4.55', 'completions/max_length': '4.6', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.588', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '4.6', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2875', 'rewards/reward_semantic/std': '0.02315', 'rewards/reward_length/mean': '-0.02075', 'rewards/reward_length/std': '0.001138', 'reward': '1.085', 'reward_std': '0.006928', 'frac_reward_zero_std': '0.9', 'kl': '2.816', 'entropy': '0.04539', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.452', 'epoch': '0.8071'}
{'loss': '0.08125', 'grad_norm': '11.91', 'learning_rate': '2.531e-06', 'num_tokens': '1.468e+06', 'completions/mean_length': '5.1', 'completions/min_length': '4.75', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.1', 'completions/min_terminated_length': '4.75', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2705', 'rewards/reward_semantic/std': '0.0465', 'rewards/reward_length/mean': '-0.04058', 'rewards/reward_length/std': '0.005178', 'reward': '1.079', 'reward_std': '0.01397', 'frac_reward_zero_std': '0.75', 'kl': '2.579', 'entropy': '0.1038', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.531', 'epoch': '0.8104'}
{'loss': '0.1165', 'grad_norm': '1.428e-07', 'learning_rate': '2.521e-06', 'num_tokens': '1.474e+06', 'completions/mean_length': '4.675', 'completions/min_length': '4.55', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.675', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2634', 'rewards/reward_semantic/std': '0.03919', 'rewards/reward_length/mean': '-0.01946', 'rewards/reward_length/std': '0.001283', 'reward': '1.078', 'reward_std': '0.01171', 'frac_reward_zero_std': '0.85', 'kl': '2.941', 'entropy': '0.07076', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.49', 'epoch': '0.8136'}
{'loss': '0.08464', 'grad_norm': '0.0004207', 'learning_rate': '2.511e-06', 'num_tokens': '1.48e+06', 'completions/mean_length': '4.775', 'completions/min_length': '4.35', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.775', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3368', 'rewards/reward_semantic/std': '0.04962', 'rewards/reward_length/mean': '-0.03806', 'rewards/reward_length/std': '0.007837', 'reward': '1.099', 'reward_std': '0.01488', 'frac_reward_zero_std': '0.65', 'kl': '2.274', 'entropy': '0.1164', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.513', 'epoch': '0.8169'}
{'loss': '0.1324', 'grad_norm': '1.139e-05', 'learning_rate': '2.501e-06', 'num_tokens': '1.486e+06', 'completions/mean_length': '4.838', 'completions/min_length': '4.45', 'completions/max_length': '5.3', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.838', 'completions/min_terminated_length': '4.45', 'completions/max_terminated_length': '5.3', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3521', 'rewards/reward_semantic/std': '0.04313', 'rewards/reward_length/mean': '-0.02986', 'rewards/reward_length/std': '0.005814', 'reward': '1.104', 'reward_std': '0.01296', 'frac_reward_zero_std': '0.75', 'kl': '2.967', 'entropy': '0.1105', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.53', 'epoch': '0.8202'}
{'loss': '0.09581', 'grad_norm': '0.0009937', 'learning_rate': '2.491e-06', 'num_tokens': '1.491e+06', 'completions/mean_length': '4.881', 'completions/min_length': '4.7', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.881', 'completions/min_terminated_length': '4.7', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.393', 'rewards/reward_semantic/std': '0.08253', 'rewards/reward_length/mean': '-0.03083', 'rewards/reward_length/std': '0.005454', 'reward': '1.116', 'reward_std': '0.02477', 'frac_reward_zero_std': '0.75', 'kl': '2.539', 'entropy': '0.09604', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.537', 'epoch': '0.8235'}
{'loss': '0.1151', 'grad_norm': '0.0003117', 'learning_rate': '2.481e-06', 'num_tokens': '1.497e+06', 'completions/mean_length': '5.438', 'completions/min_length': '4.85', 'completions/max_length': '5.7', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '5.438', 'completions/min_terminated_length': '4.85', 'completions/max_terminated_length': '5.7', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2429', 'rewards/reward_semantic/std': '0.01564', 'rewards/reward_length/mean': '-0.02952', 'rewards/reward_length/std': '0.00255', 'reward': '1.071', 'reward_std': '0.004679', 'frac_reward_zero_std': '0.75', 'kl': '2.461', 'entropy': '0.07497', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.518', 'epoch': '0.8268'}

 51%|█████     | 5060/10000 [6:05:34<6:01:37,  4.39s/it]
 51%|█████     | 5061/10000 [6:05:39<6:04:24,  4.43s/it]
 51%|█████     | 5062/10000 [6:05:43<5:57:29,  4.34s/it]
 51%|█████     | 5063/10000 [6:05:47<5:59:23,  4.37s/it]
 51%|█████     | 5064/10000 [6:05:52<6:03:51,  4.42s/it]
 51%|█████     | 5065/10000 [6:05:57<6:18:33,  4.60s/it]
 51%|█████     | 5066/10000 [6:06:01<6:03:34,  4.42s/it]
 51%|█████     | 5067/10000 [6:06:05<6:04:10,  4.43s/it]
 51%|█████     | 5068/10000 [6:06:10<6:04:35,  4.44s/it]
 51%|█████     | 5069/10000 [6:06:14<5:58:14,  4.36s/it]
 51%|█████     | 5070/10000 [6:06:18<6:00:37,  4.39s/it]
 51%|█████     | 5071/10000 [6:06:23<6:02:14,  4.41s/it]
 51%|█████     | 5072/10000 [6:06:27<6:03:21,  4.42s/it]
 51%|█████     | 5073/10000 [6:06:32<5:55:12,  4.33s/it]
 51%|█████     | 5074/10000 [6:06:36<5:57:48,  4.36s/it]
 51%|█████     | 5075/10000 [6:06:40<6:02:30,  4.42s/it]
 51%|█████     | 5076/10000 [6:06:45<5:56:05,  4.34s/it]
 51%|█████     | 5077/10000 [6:06:49<5:58:59,  4.38s/it]
 51%|█████     | 5078/10000 [6:06:54<6:01:03,  4.40s/it]
 51%|█████     | 5079/10000 [6:06:58<6:04:19,  4.44s/it]
 51%|█████     | 5080/10000 [6:07:02<6:00:51,  4.40s/it]
                                                        

 51%|█████     | 5080/10000 [6:07:02<6:00:51,  4.40s/it]
 51%|█████     | 5081/10000 [6:07:07<6:05:17,  4.46s/it]
 51%|█████     | 5082/10000 [6:07:12<6:08:43,  4.50s/it]
 51%|█████     | 5083/10000 [6:07:16<6:06:50,  4.48s/it]
 51%|█████     | 5084/10000 [6:07:21<6:09:38,  4.51s/it]
 51%|█████     | 5085/10000 [6:07:25<6:11:14,  4.53s/it]
 51%|█████     | 5086/10000 [6:07:30<6:16:45,  4.60s/it]
 51%|█████     | 5087/10000 [6:07:34<6:10:56,  4.53s/it]
 51%|█████     | 5088/10000 [6:07:39<6:13:07,  4.56s/it]
 51%|█████     | 5089/10000 [6:07:44<6:14:27,  4.57s/it]
 51%|█████     | 5090/10000 [6:07:48<6:15:26,  4.59s/it]
 51%|█████     | 5091/10000 [6:07:53<6:15:11,  4.59s/it]
 51%|█████     | 5092/10000 [6:07:57<6:14:48,  4.58s/it]
 51%|█████     | 5093/10000 [6:08:02<6:14:47,  4.58s/it]
 51%|█████     | 5094/10000 [6:08:06<6:14:31,  4.58s/it]
 51%|█████     | 5095/10000 [6:08:11<6:06:44,  4.49s/it]
 51%|█████     | 5096/10000 [6:08:15<6:08:31,  4.51s/it]
 51%|█████     | 5097/10000 [6:08:20<6:09:40,  4.52s/it]
 51%|█████     | 5098/10000 [6:08:24<6:02:46,  4.44s/it]
 51%|█████     | 5099/10000 [6:08:29<6:05:34,  4.48s/it]
 51%|█████     | 5100/10000 [6:08:33<6:07:51,  4.50s/it]
                                                        

 51%|█████     | 5100/10000 [6:08:33<6:07:51,  4.50s/it]
 51%|█████     | 5101/10000 [6:08:38<6:09:01,  4.52s/it]
 51%|█████     | 5102/10000 [6:08:42<6:02:53,  4.45s/it]
 51%|█████     | 5103/10000 [6:08:47<6:06:01,  4.48s/it]
 51%|█████     | 5104/10000 [6:08:51<6:06:16,  4.49s/it]
 51%|█████     | 5105/10000 [6:08:55<6:00:56,  4.42s/it]
 51%|█████     | 5106/10000 [6:09:00<6:04:30,  4.47s/it]
 51%|█████     | 5107/10000 [6:09:05<6:07:09,  4.50s/it]
 51%|█████     | 5108/10000 [6:09:09<6:08:27,  4.52s/it]
 51%|█████     | 5109/10000 [6:09:13<6:02:30,  4.45s/it]
 51%|█████     | 5110/10000 [6:09:18<6:05:58,  4.49s/it]
 51%|█████     | 5111/10000 [6:09:23<6:07:49,  4.51s/it]
 51%|█████     | 5112/10000 [6:09:27<6:09:15,  4.53s/it]
 51%|█████     | 5113/10000 [6:09:32<6:05:04,  4.48s/it]
 51%|█████     | 5114/10000 [6:09:36<6:07:05,  4.51s/it]
 51%|█████     | 5115/10000 [6:09:41<6:09:20,  4.54s/it]
 51%|█████     | 5116/10000 [6:09:45<6:03:48,  4.47s/it]
 51%|█████     | 5117/10000 [6:09:50<6:06:39,  4.51s/it]
 51%|█████     | 5118/10000 [6:09:54<6:08:37,  4.53s/it]
 51%|█████     | 5119/10000 [6:09:59<6:11:47,  4.57s/it]
 51%|█████     | 5120/10000 [6:10:03<6:06:45,  4.51s/it]
                                                        

 51%|█████     | 5120/10000 [6:10:03<6:06:45,  4.51s/it]
 51%|█████     | 5121/10000 [6:10:08<6:08:23,  4.53s/it]
 51%|█████     | 5122/10000 [6:10:12<6:03:59,  4.48s/it]
 51%|█████     | 5123/10000 [6:10:17<6:06:15,  4.51s/it]
 51%|█████     | 5124/10000 [6:10:21<6:01:02,  4.44s/it]
 51%|█████▏    | 5125/10000 [6:10:26<6:04:18,  4.48s/it]
 51%|█████▏    | 5126/10000 [6:10:30<6:06:55,  4.52s/it]
 51%|█████▏    | 5127/10000 [6:10:35<6:02:00,  4.46s/it]
 51%|█████▏    | 5128/10000 [6:10:39<6:07:09,  4.52s/it]
 51%|█████▏    | 5129/10000 [6:10:44<6:09:16,  4.55s/it]
 51%|█████▏    | 5130/10000 [6:10:48<6:11:10,  4.57s/it]
 51%|█████▏    | 5131/10000 [6:10:53<6:05:06,  4.50s/it]
 51%|█████▏    | 5132/10000 [6:10:57<6:08:08,  4.54s/it]
 51%|█████▏    | 5133/10000 [6:11:02<6:09:55,  4.56s/it]
 51%|█████▏    | 5134/10000 [6:11:07<6:11:27,  4.58s/it]
 51%|█████▏    | 5135/10000 [6:11:11<5:59:09,  4.43s/it]
 51%|█████▏    | 5136/10000 [6:11:15<6:03:32,  4.48s/it]
 51%|█████▏    | 5137/10000 [6:11:20<6:06:51,  4.53s/it]
 51%|█████▏    | 5138/10000 [6:11:25<6:25:43,  4.76s/it]
 51%|█████▏    | 5139/10000 [6:11:30<6:20:15,  4.69s/it]
 51%|█████▏    | 5140/10000 [6:11:34<6:17:54,  4.67s/it]
                                                        

 51%|█████▏    | 5140/10000 [6:11:34<6:17:54,  4.67s/it]
 51%|█████▏    | 5141/10000 [6:11:39<6:16:43,  4.65s/it]
 51%|█████▏    | 5142/10000 [6:11:44<6:22:08,  4.72s/it]
 51%|█████▏    | 5143/10000 [6:11:48<6:13:49,  4.62s/it]
 51%|█████▏    | 5144/10000 [6:11:53<6:13:05,  4.61s/it]
 51%|█████▏    | 5145/10000 [6:11:57<6:12:48,  4.61s/it]
 51%|█████▏    | 5146/10000 [6:12:02<6:12:31,  4.60s/it]
 51%|█████▏    | 5147/10000 [6:12:06<6:04:55,  4.51s/it]
 51%|█████▏    | 5148/10000 [6:12:11<6:07:18,  4.54s/it]
 51%|█████▏    | 5149/10000 [6:12:16<6:08:42,  4.56s/it]
 52%|█████▏    | 5150/10000 [6:12:20<6:01:22,  4.47s/it]
 52%|█████▏    | 5151/10000 [6:12:24<6:04:19,  4.51s/it]
 52%|█████▏    | 5152/10000 [6:12:29<6:05:22,  4.52s/it]
 52%|█████▏    | 5153/10000 [6:12:33<6:01:35,  4.48s/it]
 52%|█████▏    | 5154/10000 [6:12:38<6:00:15,  4.46s/it]
 52%|█████▏    | 5155/10000 [6:12:43<6:10:41,  4.59s/it]
 52%|█████▏    | 5156/10000 [6:12:47<6:10:19,  4.59s/it]
 52%|█████▏    | 5157/10000 [6:12:52<6:14:51,  4.64s/it]
 52%|█████▏    | 5158/10000 [6:12:56<6:06:03,  4.54s/it]
 52%|█████▏    | 5159/10000 [6:13:02<6:23:33,  4.75s/it]
 52%|█████▏    | 5160/10000 [6:13:06<6:16:18,  4.67s/it]
                                                        

 52%|█████▏    | 5160/10000 [6:13:06<6:16:18,  4.67s/it]
 52%|█████▏    | 5161/10000 [6:13:10<6:11:39,  4.61s/it]
 52%|█████▏    | 5162/10000 [6:13:15<6:02:23,  4.49s/it]
 52%|█████▏    | 5163/10000 [6:13:19<6:01:35,  4.49s/it]
 52%|█████▏    | 5164/10000 [6:13:24<6:01:03,  4.48s/it]
 52%|█████▏    | 5165/10000 [6:13:28<6:05:06,  4.53s/it]
 52%|█████▏    | 5166/10000 [6:13:33<6:00:53,  4.48s/it]
 52%|█████▏    | 5167/10000 [6:13:37<6:01:49,  4.49s/it]
 52%|█████▏    | 5168/10000 [6:13:42<6:04:10,  4.52s/it]
 52%|█████▏    | 5169/10000 [6:13:46<6:04:05,  4.52s/it]
 52%|█████▏    | 5170/10000 [6:13:51<5:57:55,  4.45s/it]
 52%|█████▏    | 5171/10000 [6:13:55<5:55:40,  4.42s/it]
 52%|█████▏    | 5172/10000 [6:13:59<5:54:27,  4.40s/it]
 52%|█████▏    | 5173/10000 [6:14:04<5:51:52,  4.37s/it]
 52%|█████▏    | 5174/10000 [6:14:08<5:57:21,  4.44s/it]
 52%|█████▏    | 5175/10000 [6:14:13<6:02:24,  4.51s/it]
 52%|█████▏    | 5176/10000 [6:14:17<6:03:47,  4.52s/it]
 52%|█████▏    | 5177/10000 [6:14:22<6:05:30,  4.55s/it]
 52%|█████▏    | 5178/10000 [6:14:26<5:59:27,  4.47s/it]
 52%|█████▏    | 5179/10000 [6:14:31<5:56:11,  4.43s/it]
 52%|█████▏    | 5180/10000 [6:14:35<6:01:58,  4.51s/it]
                                                        

 52%|█████▏    | 5180/10000 [6:14:35<6:01:58,  4.51s/it]
 52%|█████▏    | 5181/10000 [6:14:40<6:00:15,  4.49s/it]
 52%|█████▏    | 5182/10000 [6:14:44<6:02:57,  4.52s/it]
 52%|█████▏    | 5183/10000 [6:14:49<6:04:58,  4.55s/it]
 52%|█████▏    | 5184/10000 [6:14:54<6:14:55,  4.67s/it]
 52%|█████▏    | 5185/10000 [6:14:58<6:11:11,  4.63s/it]
 52%|█████▏    | 5186/10000 [6:15:03<6:03:55,  4.54s/it]
 52%|█████▏    | 5187/10000 [6:15:07<6:05:57,  4.56s/it]
 52%|█████▏    | 5188/10000 [6:15:12<6:01:07,  4.50s/it]
 52%|█████▏    | 5189/10000 [6:15:16<6:04:25,  4.54s/it]
 52%|█████▏    | 5190/10000 [6:15:21<6:02:03,  4.52s/it]
 52%|█████▏    | 5191/10000 [6:15:25<5:58:11,  4.47s/it]
 52%|█████▏    | 5192/10000 [6:15:30<5:55:57,  4.44s/it]
 52%|█████▏    | 5193/10000 [6:15:34<5:53:21,  4.41s/it]
 52%|█████▏    | 5194/10000 [6:15:39<5:58:33,  4.48s/it]
 52%|█████▏    | 5195/10000 [6:15:43<6:02:27,  4.53s/it]
 52%|█████▏    | 5196/10000 [6:15:48<6:03:20,  4.54s/it]
 52%|█████▏    | 5197/10000 [6:15:52<6:04:43,  4.56s/it]
 52%|█████▏    | 5198/10000 [6:15:57<6:05:51,  4.57s/it]
 52%|█████▏    | 5199/10000 [6:16:01<6:01:27,  4.52s/it]
 52%|█████▏    | 5200/10000 [6:16:06<6:07:50,  4.60s/it]
                                                        

 52%|█████▏    | 5200/10000 [6:16:06<6:07:50,  4.60s/it]
 52%|█████▏    | 5201/10000 [6:16:11<6:08:21,  4.61s/it]
 52%|█████▏    | 5202/10000 [6:16:15<6:08:29,  4.61s/it]
 52%|█████▏    | 5203/10000 [6:16:20<6:01:13,  4.52s/it]
 52%|█████▏    | 5204/10000 [6:16:24<6:01:11,  4.52s/it]
 52%|█████▏    | 5205/10000 [6:16:29<6:02:33,  4.54s/it]
 52%|█████▏    | 5206/10000 [6:16:33<6:03:50,  4.55s/it]
 52%|█████▏    | 5207/10000 [6:16:38<6:04:52,  4.57s/it]
 52%|█████▏    | 5208/10000 [6:16:42<5:58:25,  4.49s/it]
 52%|█████▏    | 5209/10000 [6:16:47<6:01:05,  4.52s/it]
 52%|█████▏    | 5210/10000 [6:16:52<6:03:01,  4.55s/it]
 52%|█████▏    | 5211/10000 [6:16:56<6:03:45,  4.56s/it]
 52%|█████▏    | 5212/10000 [6:17:00<5:58:12,  4.49s/it]
 52%|█████▏    | 5213/10000 [6:17:05<6:02:02,  4.54s/it]
 52%|█████▏    | 5214/10000 [6:17:10<6:03:31,  4.56s/it]
 52%|█████▏    | 5215/10000 [6:17:14<6:04:25,  4.57s/it]
 52%|█████▏    | 5216/10000 [6:17:19<6:09:51,  4.64s/it]
 52%|█████▏    | 5217/10000 [6:17:23<5:59:59,  4.52s/it]
 52%|█████▏    | 5218/10000 [6:17:28<6:00:07,  4.52s/it]
 52%|█████▏    | 5219/10000 [6:17:32<6:01:17,  4.53s/it]
 52%|█████▏    | 5220/10000 [6:17:37<6:04:16,  4.57s/it]
                                                        

 52%|█████▏    | 5220/10000 [6:17:37<6:04:16,  4.57s/it]
 52%|█████▏    | 5221/10000 [6:17:41<5:57:48,  4.49s/it]
 52%|█████▏    | 5222/10000 [6:17:46<5:59:44,  4.52s/it]
 52%|█████▏    | 5223/10000 [6:17:50<5:59:00,  4.51s/it]
 52%|█████▏    | 5224/10000 [6:17:55<6:00:47,  4.53s/it]
 52%|█████▏    | 5225/10000 [6:18:00<6:06:36,  4.61s/it]
 52%|█████▏    | 5226/10000 [6:18:04<6:08:29,  4.63s/it]
 52%|█████▏    | 5227/10000 [6:18:09<6:08:52,  4.64s/it]
 52%|█████▏    | 5228/10000 [6:18:14<6:08:27,  4.63s/it]
 52%|█████▏    | 5229/10000 [6:18:18<6:07:48,  4.63s/it]
 52%|█████▏    | 5230/10000 [6:18:23<6:07:17,  4.62s/it]
 52%|█████▏    | 5231/10000 [6:18:27<5:57:49,  4.50s/it]
 52%|█████▏    | 5232/10000 [6:18:32<6:00:19,  4.53s/it]
 52%|█████▏    | 5233/10000 [6:18:36<6:01:38,  4.55s/it]
 52%|█████▏    | 5234/10000 [6:18:41<5:59:32,  4.53s/it]
 52%|█████▏    | 5235/10000 [6:18:45<5:54:24,  4.46s/it]
 52%|█████▏    | 5236/10000 [6:18:49<5:47:12,  4.37s/it]
 52%|█████▏    | 5237/10000 [6:18:54<5:52:49,  4.44s/it]
 52%|█████▏    | 5238/10000 [6:18:59<5:56:34,  4.49s/it]
 52%|█████▏    | 5239/10000 [6:19:03<5:59:53,  4.54s/it]
 52%|█████▏    | 5240/10000 [6:19:07<5:50:31,  4.42s/it]
                                                        
{'loss': '0.08813', 'grad_norm': '18.65', 'learning_rate': '2.471e-06', 'num_tokens': '1.503e+06', 'completions/mean_length': '4.819', 'completions/min_length': '4.55', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.819', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3611', 'rewards/reward_semantic/std': '0.00766', 'rewards/reward_length/mean': '-0.03', 'rewards/reward_length/std': '0.0022', 'reward': '1.107', 'reward_std': '0.002231', 'frac_reward_zero_std': '0.9', 'kl': '2.792', 'entropy': '0.1174', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.391', 'epoch': '0.8301'}
{'loss': '0.06114', 'grad_norm': '0.006546', 'learning_rate': '2.46e-06', 'num_tokens': '1.509e+06', 'completions/mean_length': '4.725', 'completions/min_length': '4.4', 'completions/max_length': '5.2', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.725', 'completions/min_terminated_length': '4.4', 'completions/max_terminated_length': '5.2', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2173', 'rewards/reward_semantic/std': '0.02429', 'rewards/reward_length/mean': '-0.0293', 'rewards/reward_length/std': '0.004133', 'reward': '1.064', 'reward_std': '0.007276', 'frac_reward_zero_std': '0.75', 'kl': '2.805', 'entropy': '0.08881', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.387', 'epoch': '0.8333'}
{'loss': '0.09936', 'grad_norm': '0.003207', 'learning_rate': '2.451e-06', 'num_tokens': '1.515e+06', 'completions/mean_length': '4.7', 'completions/min_length': '4.55', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.7', 'completions/min_terminated_length': '4.55', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2809', 'rewards/reward_semantic/std': '0.06545', 'rewards/reward_length/mean': '-0.02769', 'rewards/reward_length/std': '0.003415', 'reward': '1.083', 'reward_std': '0.01949', 'frac_reward_zero_std': '0.75', 'kl': '2.94', 'entropy': '0.1007', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.521', 'epoch': '0.8366'}
{'loss': '0.12', 'grad_norm': '11.48', 'learning_rate': '2.441e-06', 'num_tokens': '1.521e+06', 'completions/mean_length': '4.963', 'completions/min_length': '4.65', 'completions/max_length': '5.05', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.963', 'completions/min_terminated_length': '4.65', 'completions/max_terminated_length': '5.05', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3775', 'rewards/reward_semantic/std': '0.08519', 'rewards/reward_length/mean': '-0.03257', 'rewards/reward_length/std': '0.005937', 'reward': '1.112', 'reward_std': '0.02544', 'frac_reward_zero_std': '0.7', 'kl': '2.811', 'entropy': '0.07535', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.478', 'epoch': '0.8399'}
{'loss': '0.1436', 'grad_norm': '0.0004749', 'learning_rate': '2.431e-06', 'num_tokens': '1.527e+06', 'completions/mean_length': '4.606', 'completions/min_length': '4.2', 'completions/max_length': '5.1', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.606', 'completions/min_terminated_length': '4.2', 'completions/max_terminated_length': '5.1', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.1804', 'rewards/reward_semantic/std': '0.002465', 'rewards/reward_length/mean': '-0.02737', 'rewards/reward_length/std': '0.004971', 'reward': '1.053', 'reward_std': '0.0009878', 'frac_reward_zero_std': '0.8', 'kl': '2.844', 'entropy': '0.1727', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.538', 'epoch': '0.8432'}
{'loss': '0.08176', 'grad_norm': '0.003431', 'learning_rate': '2.42e-06', 'num_tokens': '1.532e+06', 'completions/mean_length': '4.794', 'completions/min_length': '4.35', 'completions/max_length': '5.35', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.794', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '5.35', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2889', 'rewards/reward_semantic/std': '0.09615', 'rewards/reward_length/mean': '-0.04033', 'rewards/reward_length/std': '0.007176', 'reward': '1.085', 'reward_std': '0.02866', 'frac_reward_zero_std': '0.6', 'kl': '2.656', 'entropy': '0.1735', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.562', 'epoch': '0.8465'}
{'loss': '0.07618', 'grad_norm': '17.81', 'learning_rate': '2.411e-06', 'num_tokens': '1.538e+06', 'completions/mean_length': '4.094', 'completions/min_length': '3.8', 'completions/max_length': '4.4', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.094', 'completions/min_terminated_length': '3.8', 'completions/max_terminated_length': '4.4', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.3542', 'rewards/reward_semantic/std': '0.02252', 'rewards/reward_length/mean': '-0.02886', 'rewards/reward_length/std': '0.005897', 'reward': '1.105', 'reward_std': '0.006767', 'frac_reward_zero_std': '0.75', 'kl': '2.732', 'entropy': '0.2036', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.446', 'epoch': '0.8497'}
{'loss': '0.1213', 'grad_norm': '11.03', 'learning_rate': '2.401e-06', 'num_tokens': '1.544e+06', 'completions/mean_length': '4.231', 'completions/min_length': '3.9', 'completions/max_length': '4.65', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.231', 'completions/min_terminated_length': '3.9', 'completions/max_terminated_length': '4.65', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.4148', 'rewards/reward_semantic/std': '0.07074', 'rewards/reward_length/mean': '-0.03223', 'rewards/reward_length/std': '0.005528', 'reward': '1.123', 'reward_std': '0.02122', 'frac_reward_zero_std': '0.7', 'kl': '2.943', 'entropy': '0.1612', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.521', 'epoch': '0.853'}
{'loss': '0.108', 'grad_norm': '32.26', 'learning_rate': '2.391e-06', 'num_tokens': '1.55e+06', 'completions/mean_length': '4.644', 'completions/min_length': '4.35', 'completions/max_length': '4.85', 'completions/clipped_ratio': '0', 'completions/mean_terminated_length': '4.644', 'completions/min_terminated_length': '4.35', 'completions/max_terminated_length': '4.85', 'rewards/reward_grounded/mean': '1', 'rewards/reward_grounded/std': '0', 'rewards/reward_semantic/mean': '0.2411', 'rewards/reward_semantic/std': '0.04406', 'rewards/reward_length/mean': '-0.02688', 'rewards/reward_length/std': '0.003351', 'reward': '1.071', 'reward_std': '0.01324', 'frac_reward_zero_std': '0.75', 'kl': '2.776', 'entropy': '0.1548', 'clip_ratio/low_mean': '0', 'clip_ratio/low_min': '0', 'clip_ratio/high_mean': '0', 'clip_ratio/high_max': '0', 'clip_ratio/region_mean': '0', 'step_time': '4.525', 'epoch': '0.8563'}

 52%|█████▏    | 5240/10000 [6:19:07<5:50:31,  4.42s/it]
 52%|█████▏    | 5241/10000 [6:19:12<5:52:56,  4.45s/it]
 52%|█████▏    | 5242/10000 [6:19:17<6:01:02,  4.55s/it]
 52%|█████▏    | 5243/10000 [6:19:22<6:12:00,  4.69s/it]
 52%|█████▏    | 5244/10000 [6:19:26<6:09:28,  4.66s/it]
 52%|█████▏    | 5245/10000 [6:19:31<6:00:48,  4.55s/it]
 52%|█████▏    | 5246/10000 [6:19:35<6:01:52,  4.57s/it]
 52%|█████▏    | 5247/10000 [6:19:40<6:00:11,  4.55s/it]
 52%|█████▏    | 5248/10000 [6:19:44<6:02:47,  4.58s/it]
 52%|█████▏    | 5249/10000 [6:19:49<6:02:51,  4.58s/it]