wandb: Tracking run with wandb version 0.28.0
wandb: W&B syncing is set to `offline` in this directory. Run `wandb online` or set WANDB_MODE=online to enable cloud syncing.
wandb: Run data is saved locally in outputs/healed/opd_warm_keep50/wandb/offline-run-20260802_001724-yclwxiwa
wandb: View this run in the terminal with `wandb leet`

Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards:  33%|███▎      | 1/3 [00:00<00:00,  9.21it/s]
Loading checkpoint shards:  67%|██████▋   | 2/3 [00:00<00:00,  8.96it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:00<00:00,  9.81it/s]
teacher converted to fused MoE path (allenai/OLMoE-1B-7B-0125-Instruct)

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  6.59it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  8.19it/s]
reference anchor loaded from outputs/healed/keep50_offpolicy_warmup_s1224/step0150 on cuda:0 (beta=0.05)

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  6.63it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  8.41it/s]
starting vllm rollout server on GPU 2 (port 8377) ...
vllm server healthy in 36s
mixture distillation: 58360 gold trajectories from outputs/teacher_trajectories/dolci_combined_top128, lambda=0.5 decay=0.0

Filter:   0%|          | 0/169964 [00:00<?, ? examples/s]
Filter:   1%|          | 2000/169964 [00:00<00:09, 18189.01 examples/s]
Filter:   2%|▏         | 4000/169964 [00:00<00:09, 18391.40 examples/s]
Filter:   4%|▎         | 6000/169964 [00:00<00:08, 18333.49 examples/s]
Filter:   5%|▍         | 8000/169964 [00:00<00:08, 18616.15 examples/s]
Filter:   6%|▌         | 10000/169964 [00:00<00:08, 18611.19 examples/s]
Filter:   7%|▋         | 12000/169964 [00:00<00:08, 18757.24 examples/s]
Filter:   8%|▊         | 14000/169964 [00:00<00:08, 18765.81 examples/s]
Filter:   9%|▉         | 16000/169964 [00:00<00:08, 18657.06 examples/s]
Filter:  11%|█         | 18000/169964 [00:00<00:08, 18715.46 examples/s]
Filter:  12%|█▏        | 20000/169964 [00:01<00:07, 18870.01 examples/s]
Filter:  14%|█▍        | 24000/169964 [00:01<00:06, 22266.44 examples/s]
Filter:  16%|█▋        | 28000/169964 [00:01<00:05, 24403.47 examples/s]
Filter:  19%|█▉        | 32000/169964 [00:01<00:05, 25768.94 examples/s]
Filter:  21%|██        | 36000/169964 [00:01<00:05, 26477.46 examples/s]
Filter:  24%|██▎       | 40000/169964 [00:01<00:04, 27041.78 examples/s]
Filter:  26%|██▌       | 44000/169964 [00:01<00:04, 28286.34 examples/s]
Filter:  28%|██▊       | 48000/169964 [00:02<00:04, 29111.94 examples/s]
Filter:  31%|███       | 52000/169964 [00:02<00:03, 29856.19 examples/s]
Filter:  32%|███▏      | 55000/169964 [00:02<00:03, 28870.02 examples/s]
Filter:  34%|███▍      | 58000/169964 [00:02<00:04, 25853.16 examples/s]
Filter:  36%|███▋      | 62000/169964 [00:02<00:04, 23470.19 examples/s]
Filter:  38%|███▊      | 65000/169964 [00:02<00:04, 22321.96 examples/s]
Filter:  40%|████      | 68000/169964 [00:02<00:04, 21327.67 examples/s]
Filter:  42%|████▏     | 72000/169964 [00:03<00:04, 23162.35 examples/s]
Filter:  45%|████▍     | 76000/169964 [00:03<00:03, 24639.56 examples/s]
Filter:  47%|████▋     | 80000/169964 [00:03<00:03, 25433.16 examples/s]
Filter:  49%|████▉     | 84000/169964 [00:03<00:03, 25951.46 examples/s]
Filter:  51%|█████     | 87000/169964 [00:03<00:04, 20093.91 examples/s]
Filter:  53%|█████▎    | 90000/169964 [00:04<00:04, 17113.96 examples/s]
Filter:  54%|█████▍    | 92000/169964 [00:04<00:04, 15747.23 examples/s]
Filter:  55%|█████▌    | 94000/169964 [00:04<00:05, 14827.68 examples/s]
Filter:  56%|█████▋    | 96000/169964 [00:04<00:05, 13887.48 examples/s]
Filter:  58%|█████▊    | 98000/169964 [00:04<00:05, 13382.01 examples/s]
Filter:  59%|█████▉    | 100000/169964 [00:04<00:05, 12934.54 examples/s]
Filter:  60%|██████    | 102000/169964 [00:05<00:05, 12623.76 examples/s]
Filter:  61%|██████    | 104000/169964 [00:05<00:05, 12416.94 examples/s]
Filter:  62%|██████▏   | 106000/169964 [00:05<00:05, 12237.71 examples/s]
Filter:  64%|██████▎   | 108000/169964 [00:05<00:05, 12211.30 examples/s]
Filter:  65%|██████▍   | 110000/169964 [00:05<00:04, 12044.34 examples/s]
Filter:  66%|██████▌   | 112000/169964 [00:05<00:04, 11961.44 examples/s]
Filter:  67%|██████▋   | 114000/169964 [00:06<00:04, 11895.55 examples/s]
Filter:  68%|██████▊   | 116000/169964 [00:06<00:04, 11958.18 examples/s]
Filter:  69%|██████▉   | 118000/169964 [00:06<00:04, 11887.92 examples/s]
Filter:  71%|███████   | 120000/169964 [00:06<00:04, 11843.06 examples/s]
Filter:  72%|███████▏  | 122000/169964 [00:06<00:03, 12034.62 examples/s]
Filter:  73%|███████▎  | 124000/169964 [00:06<00:03, 13008.86 examples/s]
Filter:  74%|███████▍  | 126000/169964 [00:06<00:03, 14048.33 examples/s]
Filter:  75%|███████▌  | 128000/169964 [00:07<00:02, 14972.93 examples/s]
Filter:  76%|███████▋  | 130000/169964 [00:07<00:02, 15729.83 examples/s]
Filter:  78%|███████▊  | 132000/169964 [00:07<00:02, 16241.27 examples/s]
Filter:  79%|███████▉  | 134000/169964 [00:07<00:02, 16497.20 examples/s]
Filter:  80%|████████  | 136000/169964 [00:07<00:02, 16863.78 examples/s]
Filter:  81%|████████  | 138000/169964 [00:07<00:01, 16598.63 examples/s]
Filter:  82%|████████▏ | 140000/169964 [00:07<00:01, 16438.45 examples/s]
Filter:  84%|████████▎ | 142000/169964 [00:07<00:01, 16557.52 examples/s]
Filter:  85%|████████▍ | 144000/169964 [00:07<00:01, 16832.68 examples/s]
Filter:  86%|████████▌ | 146000/169964 [00:08<00:01, 16985.48 examples/s]
Filter:  87%|████████▋ | 148000/169964 [00:08<00:01, 17138.73 examples/s]
Filter:  88%|████████▊ | 150000/169964 [00:08<00:01, 17218.05 examples/s]
Filter:  89%|████████▉ | 152000/169964 [00:08<00:01, 17292.31 examples/s]
Filter:  91%|█████████ | 154000/169964 [00:08<00:00, 17222.19 examples/s]
Filter:  92%|█████████▏| 156000/169964 [00:08<00:00, 17206.06 examples/s]
Filter:  93%|█████████▎| 158000/169964 [00:08<00:00, 17366.63 examples/s]
Filter:  94%|█████████▍| 160000/169964 [00:08<00:00, 17278.87 examples/s]
Filter:  95%|█████████▌| 162000/169964 [00:09<00:00, 16863.79 examples/s]
Filter:  96%|█████████▋| 164000/169964 [00:09<00:00, 16736.44 examples/s]
Filter:  98%|█████████▊| 166000/169964 [00:09<00:00, 16904.54 examples/s]
Filter:  99%|█████████▉| 168000/169964 [00:09<00:00, 16811.94 examples/s]
Filter: 100%|██████████| 169964/169964 [00:09<00:00, 11290.22 examples/s]
Filter: 100%|██████████| 169964/169964 [00:09<00:00, 17511.33 examples/s]
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
168191 prompts | 2627 steps/epoch | 120 total steps | student params 3.70B | teacher overlap=True
{"step": 1, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0715529867857179, "tokens": 330677, "cumulative_loss_tokens": 330677, "grad_norm": 18.0, "lr": 2.0000000000000003e-06, "finish_rate": 0.66, "comp_len": 1291.7, "dropped_truncated": 0, "gold_loss": 0.4816, "gold_lambda": 0.5, "rep_ratio": 7.359, "t_data_s": 0.0, "t_rollout_s": 117.2, "t_step_s": 230.0, "t_refresh_s": 0.3, "mem_gb": 10.4, "mem_gb_teacher": 20.7}
[eval step 1] sample: 'To "faye" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m" "n" "m'
{"step": 1, "gsm8k_n": 256, "gsm8k_quick_chat": 0.46875, "t_eval_s": 40.3}
{"step": 2, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0476409321224727, "tokens": 305425, "cumulative_loss_tokens": 636102, "grad_norm": 19.25, "lr": 3e-06, "finish_rate": 0.699, "comp_len": 1193.1, "dropped_truncated": 0, "gold_loss": 0.4636, "gold_lambda": 0.5, "rep_ratio": 6.842, "t_data_s": 0.0, "t_rollout_s": 113.7, "t_step_s": 214.3, "t_refresh_s": 0.3, "mem_gb": 10.55, "mem_gb_teacher": 20.67}
{"step": 3, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.2240935077801478, "tokens": 302635, "cumulative_loss_tokens": 938737, "grad_norm": 16.75, "lr": 4.000000000000001e-06, "finish_rate": 0.711, "comp_len": 1182.2, "dropped_truncated": 0, "gold_loss": 0.4989, "gold_lambda": 0.5, "rep_ratio": 8.206, "t_data_s": 0.0, "t_rollout_s": 109.6, "t_step_s": 211.3, "t_refresh_s": 0.3, "mem_gb": 10.46, "mem_gb_teacher": 20.63}
{"step": 4, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.1232984279951, "tokens": 314442, "cumulative_loss_tokens": 1253179, "grad_norm": 13.625, "lr": 5e-06, "finish_rate": 0.723, "comp_len": 1228.3, "dropped_truncated": 0, "gold_loss": 0.4989, "gold_lambda": 0.5, "rep_ratio": 4.007, "t_data_s": 0.0, "t_rollout_s": 112.2, "t_step_s": 215.0, "t_refresh_s": 0.3, "mem_gb": 10.24, "mem_gb_teacher": 20.57}
{"step": 5, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.196489499796483, "tokens": 314169, "cumulative_loss_tokens": 1567348, "grad_norm": 12.0625, "lr": 6e-06, "finish_rate": 0.715, "comp_len": 1227.2, "dropped_truncated": 0, "gold_loss": 1.7581, "gold_lambda": 0.5, "rep_ratio": 7.457, "t_data_s": 0.0, "t_rollout_s": 113.5, "t_step_s": 213.6, "t_refresh_s": 0.3, "mem_gb": 10.48, "mem_gb_teacher": 20.64}
{"step": 6, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.174327937240284, "tokens": 302861, "cumulative_loss_tokens": 1870209, "grad_norm": 10.125, "lr": 7e-06, "finish_rate": 0.715, "comp_len": 1183.1, "dropped_truncated": 0, "gold_loss": 1.4087, "gold_lambda": 0.5, "rep_ratio": 6.804, "t_data_s": 0.0, "t_rollout_s": 112.3, "t_step_s": 214.1, "t_refresh_s": 0.3, "mem_gb": 10.49, "mem_gb_teacher": 20.64}
{"step": 7, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.142888064626811, "tokens": 259225, "cumulative_loss_tokens": 2129434, "grad_norm": 8.0625, "lr": 8.000000000000001e-06, "finish_rate": 0.875, "comp_len": 1012.6, "dropped_truncated": 0, "gold_loss": 1.7253, "gold_lambda": 0.5, "rep_ratio": 2.291, "t_data_s": 0.0, "t_rollout_s": 85.8, "t_step_s": 180.9, "t_refresh_s": 0.3, "mem_gb": 10.5, "mem_gb_teacher": 20.66}
{"step": 8, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.2367885471050992, "tokens": 275165, "cumulative_loss_tokens": 2404599, "grad_norm": 10.0625, "lr": 9e-06, "finish_rate": 0.777, "comp_len": 1074.9, "dropped_truncated": 0, "gold_loss": 1.791, "gold_lambda": 0.5, "rep_ratio": 4.278, "t_data_s": 0.0, "t_rollout_s": 101.9, "t_step_s": 198.9, "t_refresh_s": 0.3, "mem_gb": 10.37, "mem_gb_teacher": 20.6}
{"step": 9, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.2960635659850794, "tokens": 287956, "cumulative_loss_tokens": 2692555, "grad_norm": 10.5, "lr": 1e-05, "finish_rate": 0.777, "comp_len": 1124.8, "dropped_truncated": 0, "gold_loss": 0.4427, "gold_lambda": 0.5, "rep_ratio": 3.539, "t_data_s": 0.0, "t_rollout_s": 107.3, "t_step_s": 217.4, "t_refresh_s": 0.3, "mem_gb": 10.37, "mem_gb_teacher": 20.61}
{"step": 10, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.1268998827677594, "tokens": 256196, "cumulative_loss_tokens": 2948751, "grad_norm": 8.1875, "lr": 1e-05, "finish_rate": 0.852, "comp_len": 1000.8, "dropped_truncated": 0, "gold_loss": 0.4179, "gold_lambda": 0.5, "rep_ratio": 5.761, "t_data_s": 0.0, "t_rollout_s": 83.3, "t_step_s": 177.5, "t_refresh_s": 0.3, "mem_gb": 10.63, "mem_gb_teacher": 20.7}
[eval step 10] sample: 'To be a-m-on-n-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-on-'
{"step": 11, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.1852989046363867, "tokens": 249794, "cumulative_loss_tokens": 3198545, "grad_norm": 9.4375, "lr": 1e-05, "finish_rate": 0.879, "comp_len": 975.8, "dropped_truncated": 0, "gold_loss": 1.8732, "gold_lambda": 0.5, "rep_ratio": 2.592, "t_data_s": 0.0, "t_rollout_s": 79.1, "t_step_s": 170.9, "t_refresh_s": 0.3, "mem_gb": 10.17, "mem_gb_teacher": 20.53}
{"step": 12, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0672082473087046, "tokens": 235329, "cumulative_loss_tokens": 3433874, "grad_norm": 6.40625, "lr": 1e-05, "finish_rate": 0.918, "comp_len": 919.3, "dropped_truncated": 0, "gold_loss": 1.7626, "gold_lambda": 0.5, "rep_ratio": 2.465, "t_data_s": 0.0, "t_rollout_s": 72.1, "t_step_s": 161.9, "t_refresh_s": 0.3, "mem_gb": 10.51, "mem_gb_teacher": 20.65}
{"step": 13, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.1896850641550707, "tokens": 196953, "cumulative_loss_tokens": 3630827, "grad_norm": 6.78125, "lr": 1e-05, "finish_rate": 0.945, "comp_len": 769.3, "dropped_truncated": 0, "gold_loss": 1.8544, "gold_lambda": 0.5, "rep_ratio": 2.39, "t_data_s": 0.0, "t_rollout_s": 57.2, "t_step_s": 138.9, "t_refresh_s": 0.3, "mem_gb": 10.31, "mem_gb_teacher": 20.59}
{"step": 14, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.110568549902207, "tokens": 195354, "cumulative_loss_tokens": 3826181, "grad_norm": 5.25, "lr": 1e-05, "finish_rate": 0.938, "comp_len": 763.1, "dropped_truncated": 0, "gold_loss": 1.7907, "gold_lambda": 0.5, "rep_ratio": 2.662, "t_data_s": 0.0, "t_rollout_s": 57.3, "t_step_s": 140.7, "t_refresh_s": 0.3, "mem_gb": 10.2, "mem_gb_teacher": 20.55}
{"step": 15, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0880019511972423, "tokens": 211631, "cumulative_loss_tokens": 4037812, "grad_norm": 4.78125, "lr": 1e-05, "finish_rate": 0.938, "comp_len": 826.7, "dropped_truncated": 0, "gold_loss": 0.4573, "gold_lambda": 0.5, "rep_ratio": 4.91, "t_data_s": 0.0, "t_rollout_s": 61.5, "t_step_s": 147.1, "t_refresh_s": 0.3, "mem_gb": 10.26, "mem_gb_teacher": 20.53}
{"step": 16, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9728942313239658, "tokens": 194169, "cumulative_loss_tokens": 4231981, "grad_norm": 7.90625, "lr": 1e-05, "finish_rate": 0.953, "comp_len": 758.5, "dropped_truncated": 0, "gold_loss": 0.4647, "gold_lambda": 0.5, "rep_ratio": 12.498, "t_data_s": 0.0, "t_rollout_s": 57.5, "t_step_s": 140.4, "t_refresh_s": 0.3, "mem_gb": 10.38, "mem_gb_teacher": 20.56}
{"step": 17, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0340790669212048, "tokens": 182968, "cumulative_loss_tokens": 4414949, "grad_norm": 8.5625, "lr": 1e-05, "finish_rate": 0.945, "comp_len": 714.7, "dropped_truncated": 0, "gold_loss": 1.5083, "gold_lambda": 0.5, "rep_ratio": 2.658, "t_data_s": 0.0, "t_rollout_s": 55.6, "t_step_s": 137.3, "t_refresh_s": 0.3, "mem_gb": 10.54, "mem_gb_teacher": 20.66}
{"step": 18, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9951681053661686, "tokens": 192973, "cumulative_loss_tokens": 4607922, "grad_norm": 4.09375, "lr": 1e-05, "finish_rate": 0.965, "comp_len": 753.8, "dropped_truncated": 0, "gold_loss": 1.5254, "gold_lambda": 0.5, "rep_ratio": 2.551, "t_data_s": 0.0, "t_rollout_s": 58.0, "t_step_s": 139.2, "t_refresh_s": 0.3, "mem_gb": 10.28, "mem_gb_teacher": 20.58}
{"step": 19, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.941273376354514, "tokens": 172849, "cumulative_loss_tokens": 4780771, "grad_norm": 3.65625, "lr": 1e-05, "finish_rate": 0.969, "comp_len": 675.2, "dropped_truncated": 0, "gold_loss": 1.7476, "gold_lambda": 0.5, "rep_ratio": 2.772, "t_data_s": 0.0, "t_rollout_s": 52.5, "t_step_s": 129.2, "t_refresh_s": 0.3, "mem_gb": 10.44, "mem_gb_teacher": 20.63}
{"step": 20, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8855722967075427, "tokens": 178588, "cumulative_loss_tokens": 4959359, "grad_norm": 3.34375, "lr": 1e-05, "finish_rate": 0.969, "comp_len": 697.6, "dropped_truncated": 0, "gold_loss": 1.8831, "gold_lambda": 0.5, "rep_ratio": 2.626, "t_data_s": 0.0, "t_rollout_s": 53.1, "t_step_s": 129.8, "t_refresh_s": 0.3, "mem_gb": 10.64, "mem_gb_teacher": 20.63}
[eval step 20] sample: 'To be a a "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the "m" of the'
{"step": 20, "gsm8k_n": 256, "gsm8k_quick_chat": 0.49609375, "t_eval_s": 43.9}
{"step": 21, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 1.0167943671227804, "tokens": 170868, "cumulative_loss_tokens": 5130227, "grad_norm": 4.125, "lr": 1e-05, "finish_rate": 0.988, "comp_len": 667.5, "dropped_truncated": 0, "gold_loss": 0.4302, "gold_lambda": 0.5, "rep_ratio": 2.259, "t_data_s": 0.0, "t_rollout_s": 50.9, "t_step_s": 126.6, "t_refresh_s": 0.3, "mem_gb": 10.41, "mem_gb_teacher": 20.55}
{"step": 22, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9240483859857905, "tokens": 186012, "cumulative_loss_tokens": 5316239, "grad_norm": 3.453125, "lr": 1e-05, "finish_rate": 0.973, "comp_len": 726.6, "dropped_truncated": 0, "gold_loss": 0.4496, "gold_lambda": 0.5, "rep_ratio": 2.576, "t_data_s": 0.0, "t_rollout_s": 54.9, "t_step_s": 133.4, "t_refresh_s": 0.3, "mem_gb": 10.37, "mem_gb_teacher": 20.58}
{"step": 23, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9889739579658967, "tokens": 161101, "cumulative_loss_tokens": 5477340, "grad_norm": 3.6875, "lr": 1e-05, "finish_rate": 0.973, "comp_len": 629.3, "dropped_truncated": 0, "gold_loss": 1.1295, "gold_lambda": 0.5, "rep_ratio": 2.562, "t_data_s": 0.0, "t_rollout_s": 49.2, "t_step_s": 126.1, "t_refresh_s": 0.3, "mem_gb": 10.19, "mem_gb_teacher": 20.54}
{"step": 24, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9001673450655785, "tokens": 172167, "cumulative_loss_tokens": 5649507, "grad_norm": 2.84375, "lr": 1e-05, "finish_rate": 0.98, "comp_len": 672.5, "dropped_truncated": 0, "gold_loss": 1.184, "gold_lambda": 0.5, "rep_ratio": 2.392, "t_data_s": 0.0, "t_rollout_s": 51.2, "t_step_s": 125.6, "t_refresh_s": 0.3, "mem_gb": 10.42, "mem_gb_teacher": 20.61}
{"step": 25, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8294563550778241, "tokens": 193603, "cumulative_loss_tokens": 5843110, "grad_norm": 3.203125, "lr": 1e-05, "finish_rate": 0.949, "comp_len": 756.3, "dropped_truncated": 0, "gold_loss": 2.1916, "gold_lambda": 0.5, "rep_ratio": 2.737, "t_data_s": 0.0, "t_rollout_s": 57.4, "t_step_s": 137.7, "t_refresh_s": 0.3, "mem_gb": 10.55, "mem_gb_teacher": 20.67}
{"step": 26, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8520757204397251, "tokens": 184343, "cumulative_loss_tokens": 6027453, "grad_norm": 2.6875, "lr": 1e-05, "finish_rate": 0.969, "comp_len": 720.1, "dropped_truncated": 0, "gold_loss": 1.8995, "gold_lambda": 0.5, "rep_ratio": 2.86, "t_data_s": 0.0, "t_rollout_s": 54.2, "t_step_s": 131.3, "t_refresh_s": 0.3, "mem_gb": 10.35, "mem_gb_teacher": 20.59}
{"step": 27, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9107435231664042, "tokens": 190215, "cumulative_loss_tokens": 6217668, "grad_norm": 2.625, "lr": 1e-05, "finish_rate": 0.953, "comp_len": 743.0, "dropped_truncated": 0, "gold_loss": 1.4011, "gold_lambda": 0.5, "rep_ratio": 2.559, "t_data_s": 0.0, "t_rollout_s": 55.9, "t_step_s": 134.6, "t_refresh_s": 0.3, "mem_gb": 10.41, "mem_gb_teacher": 20.62}
{"step": 28, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9077256934040764, "tokens": 178659, "cumulative_loss_tokens": 6396327, "grad_norm": 2.1875, "lr": 1e-05, "finish_rate": 0.977, "comp_len": 697.9, "dropped_truncated": 0, "gold_loss": 1.4574, "gold_lambda": 0.5, "rep_ratio": 2.687, "t_data_s": 0.0, "t_rollout_s": 51.5, "t_step_s": 126.5, "t_refresh_s": 0.3, "mem_gb": 10.37, "mem_gb_teacher": 20.54}
{"step": 29, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9122361920726376, "tokens": 179195, "cumulative_loss_tokens": 6575522, "grad_norm": 1.7109375, "lr": 1e-05, "finish_rate": 0.949, "comp_len": 700.0, "dropped_truncated": 0, "gold_loss": 0.4425, "gold_lambda": 0.5, "rep_ratio": 2.463, "t_data_s": 0.0, "t_rollout_s": 54.0, "t_step_s": 131.5, "t_refresh_s": 0.3, "mem_gb": 10.38, "mem_gb_teacher": 20.6}
{"step": 30, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8639218128417365, "tokens": 196883, "cumulative_loss_tokens": 6772405, "grad_norm": 1.625, "lr": 1e-05, "finish_rate": 0.973, "comp_len": 769.1, "dropped_truncated": 0, "gold_loss": 0.4074, "gold_lambda": 0.5, "rep_ratio": 2.545, "t_data_s": 0.0, "t_rollout_s": 57.7, "t_step_s": 138.1, "t_refresh_s": 0.3, "mem_gb": 10.48, "mem_gb_teacher": 20.64}
[eval step 30] sample: 'To solve the problem of determining the number of days in a month that has a positive count, we need to analyze the data provided and identify the pattern.\n\n**Step 1: Analyze the Data**\n\nThe data prov'
{"step": 31, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8510895792700558, "tokens": 203228, "cumulative_loss_tokens": 6975633, "grad_norm": 2.484375, "lr": 1e-05, "finish_rate": 0.98, "comp_len": 793.9, "dropped_truncated": 0, "gold_loss": 1.7625, "gold_lambda": 0.5, "rep_ratio": 2.537, "t_data_s": 0.0, "t_rollout_s": 59.8, "t_step_s": 139.0, "t_refresh_s": 0.3, "mem_gb": 10.1, "mem_gb_teacher": 20.51}
{"step": 32, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.799281217758318, "tokens": 186923, "cumulative_loss_tokens": 7162556, "grad_norm": 2.125, "lr": 1e-05, "finish_rate": 0.969, "comp_len": 730.2, "dropped_truncated": 0, "gold_loss": 1.4991, "gold_lambda": 0.5, "rep_ratio": 2.655, "t_data_s": 0.0, "t_rollout_s": 55.1, "t_step_s": 134.5, "t_refresh_s": 0.3, "mem_gb": 10.43, "mem_gb_teacher": 20.6}
{"step": 33, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.9108344851361305, "tokens": 206685, "cumulative_loss_tokens": 7369241, "grad_norm": 2.609375, "lr": 1e-05, "finish_rate": 0.941, "comp_len": 807.4, "dropped_truncated": 0, "gold_loss": 1.8247, "gold_lambda": 0.5, "rep_ratio": 2.723, "t_data_s": 0.0, "t_rollout_s": 61.8, "t_step_s": 145.9, "t_refresh_s": 0.3, "mem_gb": 10.34, "mem_gb_teacher": 20.6}
{"step": 34, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8392004683972054, "tokens": 200636, "cumulative_loss_tokens": 7569877, "grad_norm": 5.21875, "lr": 1e-05, "finish_rate": 0.953, "comp_len": 783.7, "dropped_truncated": 0, "gold_loss": 2.3876, "gold_lambda": 0.5, "rep_ratio": 2.624, "t_data_s": 0.0, "t_rollout_s": 59.5, "t_step_s": 140.6, "t_refresh_s": 0.3, "mem_gb": 10.38, "mem_gb_teacher": 20.56}
{"step": 35, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8846296758133513, "tokens": 180440, "cumulative_loss_tokens": 7750317, "grad_norm": 2.015625, "lr": 1e-05, "finish_rate": 0.98, "comp_len": 704.8, "dropped_truncated": 0, "gold_loss": 0.3641, "gold_lambda": 0.5, "rep_ratio": 2.604, "t_data_s": 0.0, "t_rollout_s": 51.9, "t_step_s": 127.8, "t_refresh_s": 0.3, "mem_gb": 10.24, "mem_gb_teacher": 20.53}
{"step": 36, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.8721849152298049, "tokens": 194494, "cumulative_loss_tokens": 7944811, "grad_norm": 1.859375, "lr": 1e-05, "finish_rate": 0.957, "comp_len": 759.7, "dropped_truncated": 0, "gold_loss": 0.4256, "gold_lambda": 0.5, "rep_ratio": 2.591, "t_data_s": 0.0, "t_rollout_s": 56.4, "t_step_s": 136.2, "t_refresh_s": 0.3, "mem_gb": 10.27, "mem_gb_teacher": 20.57}
{"step": 37, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.849565157371755, "tokens": 186626, "cumulative_loss_tokens": 8131437, "grad_norm": 2.390625, "lr": 1e-05, "finish_rate": 0.977, "comp_len": 729.0, "dropped_truncated": 0, "gold_loss": 1.6035, "gold_lambda": 0.5, "rep_ratio": 2.528, "t_data_s": 0.0, "t_rollout_s": 54.0, "t_step_s": 132.0, "t_refresh_s": 0.3, "mem_gb": 10.26, "mem_gb_teacher": 20.53}
{"step": 38, "epoch": 0, "training_mode": "on-policy", "reverse_kl": 0.7732366005655155, "tokens": 210982, "cumulative_loss_tokens": 8342419, "grad_norm": 4.96875, "lr": 1e-05, "finish_rate": 0.969, "comp_len": 824.1, "dropped_truncated": 0, "gold_loss": 1.5962, "gold_lambda": 0.5, "rep_ratio": 2.4, "t_data_s": 0.0, "t_rollout_s": 63.0, "t_step_s": 146.4, "t_refresh_s": 0.3, "mem_gb": 10.31, "mem_gb_teacher": 20.56}