| wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from /home/henry/.netrc. |
| wandb: Currently logged in as: hbfreed to https://api.wandb.ai. Use `wandb login |
| wandb: Tracking run with wandb version 0.28.0 |
| wandb: Run data is saved locally in outputs/healed/keep50_warmup_fixed_s1224/wandb/run-20260802_025702-sep45w2j |
| wandb: Run `wandb offline` to turn off syncing. |
| wandb: Syncing run warmup-fixed-keep50-s1224 |
| wandb: ⭐️ View project at https://wandb.ai/hbfreed/glean-heal |
| wandb: 🚀 View run at https://wandb.ai/hbfreed/glean-heal/runs/sep45w2j |
|
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 5.84it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 7.90it/s] |
| 12115 cached top-128 chat trajectories / 6,476,634 unique tokens | 53 steps/epoch | 150 total steps | student params 3.70B | teacher overlap=False |
| {"step": 1, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.21896971870896717, "tokens": 120000, "cumulative_loss_tokens": 120000, "grad_norm": 4.6875, "lr": 6e-06, "finish_rate": 0.907, "comp_len": 508.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.337, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 99.1, "frames": {"chat": 236}, "mem_gb": 15.78, "mem_gb_teacher": 15.78} |
| The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results. |
| [eval step 1] sample: 'To solve the given system of equations:\n\\[\n\\begin{align*}\na + b &= k, \\\\\nk + m &= p, \\\\\np + a &= r, \\\\\nb + m + r &= 18,\n\\end{align*}\n\\]\nwe need to determine the values' |
| {"step": 2, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.27215903437460465, "tokens": 120000, "cumulative_loss_tokens": 240000, "grad_norm": 4.84375, "lr": 9e-06, "finish_rate": 0.781, "comp_len": 558.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.474, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.6, "frames": {"chat": 215}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 3, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.2780314308715363, "tokens": 120000, "cumulative_loss_tokens": 360000, "grad_norm": 4.09375, "lr": 1.2e-05, "finish_rate": 0.825, "comp_len": 553.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.376, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.6, "frames": {"chat": 217}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| {"step": 4, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.21216022065331538, "tokens": 120000, "cumulative_loss_tokens": 480000, "grad_norm": 2.546875, "lr": 1.5e-05, "finish_rate": 0.8, "comp_len": 585.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.366, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.0, "frames": {"chat": 205}, "mem_gb": 15.99, "mem_gb_teacher": 15.99} |
| {"step": 5, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.13948054732351253, "tokens": 120000, "cumulative_loss_tokens": 600000, "grad_norm": 1.6484375, "lr": 1.8e-05, "finish_rate": 0.834, "comp_len": 524.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.318, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 94.5, "frames": {"chat": 229}, "mem_gb": 15.96, "mem_gb_teacher": 15.96} |
| {"step": 6, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.2525411546646928, "tokens": 120000, "cumulative_loss_tokens": 720000, "grad_norm": 2.578125, "lr": 2.1e-05, "finish_rate": 0.812, "comp_len": 538.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.336, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.0, "frames": {"chat": 223}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 7, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.1408244575532774, "tokens": 120000, "cumulative_loss_tokens": 840000, "grad_norm": 1.171875, "lr": 2.4e-05, "finish_rate": 0.708, "comp_len": 594.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.42, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.3, "frames": {"chat": 202}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| {"step": 8, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.16170320059585697, "tokens": 120000, "cumulative_loss_tokens": 960000, "grad_norm": 1.171875, "lr": 2.7000000000000002e-05, "finish_rate": 0.77, "comp_len": 574.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.453, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.6, "frames": {"chat": 209}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 9, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.1060452919805112, "tokens": 120000, "cumulative_loss_tokens": 1080000, "grad_norm": 0.703125, "lr": 3e-05, "finish_rate": 0.885, "comp_len": 528.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.397, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.7, "frames": {"chat": 227}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 10, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.11166963671234746, "tokens": 120000, "cumulative_loss_tokens": 1200000, "grad_norm": 0.66796875, "lr": 3e-05, "finish_rate": 0.848, "comp_len": 521.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.546, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.5, "frames": {"chat": 230}, "mem_gb": 16.09, "mem_gb_teacher": 16.09} |
| [eval step 10] sample: "To solve this system of equations, we need to determine the values of \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\) given the constraints that each letter represents a non-zero digit.\n\nLet's break down the pr" |
| {"step": 11, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.09883926218959192, "tokens": 120000, "cumulative_loss_tokens": 1320000, "grad_norm": 0.671875, "lr": 3e-05, "finish_rate": 0.879, "comp_len": 519.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.426, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 94.7, "frames": {"chat": 231}, "mem_gb": 15.94, "mem_gb_teacher": 15.94} |
| {"step": 12, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.09610702018613616, "tokens": 120000, "cumulative_loss_tokens": 1440000, "grad_norm": 0.55078125, "lr": 3e-05, "finish_rate": 0.882, "comp_len": 489.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.246, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 99.3, "frames": {"chat": 245}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 13, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.09665392311389248, "tokens": 120000, "cumulative_loss_tokens": 1560000, "grad_norm": 0.5703125, "lr": 3e-05, "finish_rate": 0.81, "comp_len": 571.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.375, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.0, "frames": {"chat": 210}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 14, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.1129512736000431, "tokens": 120000, "cumulative_loss_tokens": 1680000, "grad_norm": 0.578125, "lr": 3e-05, "finish_rate": 0.758, "comp_len": 568.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.312, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.4, "frames": {"chat": 211}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 15, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.09338119786353782, "tokens": 120000, "cumulative_loss_tokens": 1800000, "grad_norm": 0.453125, "lr": 3e-05, "finish_rate": 0.819, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.413, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.0, "frames": {"chat": 221}, "mem_gb": 16.08, "mem_gb_teacher": 16.08} |
| {"step": 16, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.08123978671409811, "tokens": 120000, "cumulative_loss_tokens": 1920000, "grad_norm": 0.490234375, "lr": 3e-05, "finish_rate": 0.912, "comp_len": 480.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.366, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 101.9, "frames": {"chat": 250}, "mem_gb": 15.89, "mem_gb_teacher": 15.89} |
| {"step": 17, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.08966803371421993, "tokens": 120000, "cumulative_loss_tokens": 2040000, "grad_norm": 0.49609375, "lr": 3e-05, "finish_rate": 0.79, "comp_len": 524.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.402, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.6, "frames": {"chat": 229}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 18, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06860631760672356, "tokens": 120000, "cumulative_loss_tokens": 2160000, "grad_norm": 0.408203125, "lr": 3e-05, "finish_rate": 0.888, "comp_len": 480.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.479, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 101.4, "frames": {"chat": 250}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 19, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.08370754941549773, "tokens": 120000, "cumulative_loss_tokens": 2280000, "grad_norm": 0.421875, "lr": 3e-05, "finish_rate": 0.844, "comp_len": 519.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.363, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.2, "frames": {"chat": 231}, "mem_gb": 15.91, "mem_gb_teacher": 15.91} |
| {"step": 20, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07473204600410537, "tokens": 120000, "cumulative_loss_tokens": 2400000, "grad_norm": 0.41796875, "lr": 3e-05, "finish_rate": 0.844, "comp_len": 535.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.433, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.5, "frames": {"chat": 224}, "mem_gb": 15.96, "mem_gb_teacher": 15.96} |
| [eval step 20] sample: 'To solve the system of equations given:\n\n\\[\n\\begin{align*}\na + b &= k \\\\\nk + m &= p \\\\\np + a &= r \\\\\nb + m + r &= 18\n\\end{align*}\n\\]\n\nwe need to determine the values of \\(' |
| {"step": 21, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.08024157820896556, "tokens": 120000, "cumulative_loss_tokens": 2520000, "grad_norm": 0.486328125, "lr": 3e-05, "finish_rate": 0.802, "comp_len": 566.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.329, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.4, "frames": {"chat": 212}, "mem_gb": 16.0, "mem_gb_teacher": 16.0} |
| {"step": 22, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0721184289892825, "tokens": 120000, "cumulative_loss_tokens": 2640000, "grad_norm": 0.435546875, "lr": 3e-05, "finish_rate": 0.87, "comp_len": 504.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.376, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 97.3, "frames": {"chat": 238}, "mem_gb": 15.95, "mem_gb_teacher": 15.95} |
| {"step": 23, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07248173923180438, "tokens": 120000, "cumulative_loss_tokens": 2760000, "grad_norm": 0.443359375, "lr": 3e-05, "finish_rate": 0.903, "comp_len": 466.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.445, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 103.5, "frames": {"chat": 257}, "mem_gb": 15.83, "mem_gb_teacher": 15.83} |
| {"step": 24, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0655906916304181, "tokens": 120000, "cumulative_loss_tokens": 2880000, "grad_norm": 0.36328125, "lr": 3e-05, "finish_rate": 0.868, "comp_len": 528.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.405, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.3, "frames": {"chat": 227}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 25, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07533252080177578, "tokens": 120000, "cumulative_loss_tokens": 3000000, "grad_norm": 0.380859375, "lr": 3e-05, "finish_rate": 0.838, "comp_len": 526.3, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.359, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.6, "frames": {"chat": 228}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 26, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07692283792655605, "tokens": 120000, "cumulative_loss_tokens": 3120000, "grad_norm": 0.40625, "lr": 3e-05, "finish_rate": 0.803, "comp_len": 515.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.542, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 97.0, "frames": {"chat": 233}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 27, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06803224476923546, "tokens": 120000, "cumulative_loss_tokens": 3240000, "grad_norm": 0.3828125, "lr": 3e-05, "finish_rate": 0.863, "comp_len": 515.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.351, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.2, "frames": {"chat": 233}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 28, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.09600000411309302, "tokens": 120000, "cumulative_loss_tokens": 3360000, "grad_norm": 0.46484375, "lr": 3e-05, "finish_rate": 0.731, "comp_len": 609.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.388, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 85.0, "frames": {"chat": 197}, "mem_gb": 16.13, "mem_gb_teacher": 16.13} |
| {"step": 29, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0838949200007754, "tokens": 120000, "cumulative_loss_tokens": 3480000, "grad_norm": 0.404296875, "lr": 3e-05, "finish_rate": 0.862, "comp_len": 502.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.384, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.3, "frames": {"chat": 239}, "mem_gb": 15.88, "mem_gb_teacher": 15.88} |
| {"step": 30, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.08550032369385784, "tokens": 120000, "cumulative_loss_tokens": 3600000, "grad_norm": 0.640625, "lr": 3e-05, "finish_rate": 0.83, "comp_len": 535.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.302, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.8, "frames": {"chat": 224}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| [eval step 30] sample: 'To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to follow these steps:\n\n1. **Understand the Equations:**\n \\[\n \\begin{align*}\n a + b &= k \\\\' |
| {"step": 31, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06893708595448794, "tokens": 120000, "cumulative_loss_tokens": 3720000, "grad_norm": 0.3828125, "lr": 3e-05, "finish_rate": 0.788, "comp_len": 553.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.388, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.4, "frames": {"chat": 217}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 32, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0662639382578743, "tokens": 120000, "cumulative_loss_tokens": 3840000, "grad_norm": 0.376953125, "lr": 3e-05, "finish_rate": 0.876, "comp_len": 497.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.259, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.4, "frames": {"chat": 241}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 33, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06439438750580885, "tokens": 120000, "cumulative_loss_tokens": 3960000, "grad_norm": 0.39453125, "lr": 3e-05, "finish_rate": 0.835, "comp_len": 550.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.407, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.8, "frames": {"chat": 218}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 34, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06970151182319968, "tokens": 120000, "cumulative_loss_tokens": 4080000, "grad_norm": 0.353515625, "lr": 3e-05, "finish_rate": 0.767, "comp_len": 582.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.439, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.2, "frames": {"chat": 206}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 35, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06962556957538861, "tokens": 120000, "cumulative_loss_tokens": 4200000, "grad_norm": 0.37890625, "lr": 3e-05, "finish_rate": 0.845, "comp_len": 517.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.512, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.4, "frames": {"chat": 232}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| {"step": 36, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07808773103132843, "tokens": 120000, "cumulative_loss_tokens": 4320000, "grad_norm": 0.419921875, "lr": 3e-05, "finish_rate": 0.771, "comp_len": 550.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.432, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.0, "frames": {"chat": 218}, "mem_gb": 16.09, "mem_gb_teacher": 16.09} |
| {"step": 37, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07417992857682208, "tokens": 120000, "cumulative_loss_tokens": 4440000, "grad_norm": 0.376953125, "lr": 3e-05, "finish_rate": 0.779, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.428, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.9, "frames": {"chat": 213}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 38, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07201246263841167, "tokens": 120000, "cumulative_loss_tokens": 4560000, "grad_norm": 0.369140625, "lr": 3e-05, "finish_rate": 0.887, "comp_len": 483.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.527, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 100.9, "frames": {"chat": 248}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 39, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05988815330729509, "tokens": 120000, "cumulative_loss_tokens": 4680000, "grad_norm": 0.373046875, "lr": 3e-05, "finish_rate": 0.803, "comp_len": 550.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.333, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.1, "frames": {"chat": 218}, "mem_gb": 16.08, "mem_gb_teacher": 16.08} |
| {"step": 40, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05963528015368308, "tokens": 120000, "cumulative_loss_tokens": 4800000, "grad_norm": 0.3359375, "lr": 3e-05, "finish_rate": 0.851, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.401, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.5, "frames": {"chat": 221}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| [eval step 40] sample: "To solve the given system of equations, we need to determine the values of \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\) such that each letter represents a non-zero digit. Let's break down the problem step-by" |
| {"step": 41, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05816531496203194, "tokens": 120000, "cumulative_loss_tokens": 4920000, "grad_norm": 0.345703125, "lr": 3e-05, "finish_rate": 0.894, "comp_len": 508.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.396, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.1, "frames": {"chat": 236}, "mem_gb": 15.97, "mem_gb_teacher": 15.97} |
| {"step": 42, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06005277933338657, "tokens": 120000, "cumulative_loss_tokens": 5040000, "grad_norm": 0.326171875, "lr": 3e-05, "finish_rate": 0.858, "comp_len": 487.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.338, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 100.6, "frames": {"chat": 246}, "mem_gb": 15.89, "mem_gb_teacher": 15.89} |
| {"step": 43, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06171362210111692, "tokens": 120000, "cumulative_loss_tokens": 5160000, "grad_norm": 0.35546875, "lr": 3e-05, "finish_rate": 0.838, "comp_len": 512.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.48, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.2, "frames": {"chat": 234}, "mem_gb": 16.14, "mem_gb_teacher": 16.14} |
| {"step": 44, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.055975045030827945, "tokens": 120000, "cumulative_loss_tokens": 5280000, "grad_norm": 0.3515625, "lr": 3e-05, "finish_rate": 0.748, "comp_len": 594.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.474, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 85.4, "frames": {"chat": 202}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 45, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06047188884726105, "tokens": 120000, "cumulative_loss_tokens": 5400000, "grad_norm": 0.3359375, "lr": 3e-05, "finish_rate": 0.811, "comp_len": 553.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.36, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.3, "frames": {"chat": 217}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 46, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05655016646341731, "tokens": 120000, "cumulative_loss_tokens": 5520000, "grad_norm": 0.357421875, "lr": 3e-05, "finish_rate": 0.866, "comp_len": 535.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.47, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.3, "frames": {"chat": 224}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 47, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0790889122961089, "tokens": 120000, "cumulative_loss_tokens": 5640000, "grad_norm": 0.828125, "lr": 3e-05, "finish_rate": 0.753, "comp_len": 558.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.285, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.5, "frames": {"chat": 215}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 48, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05377363468687981, "tokens": 120000, "cumulative_loss_tokens": 5760000, "grad_norm": 0.353515625, "lr": 3e-05, "finish_rate": 0.884, "comp_len": 463.3, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.356, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 104.3, "frames": {"chat": 259}, "mem_gb": 15.97, "mem_gb_teacher": 15.97} |
| {"step": 49, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.06014201375305032, "tokens": 120000, "cumulative_loss_tokens": 5880000, "grad_norm": 0.3359375, "lr": 3e-05, "finish_rate": 0.829, "comp_len": 571.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.475, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.9, "frames": {"chat": 210}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 50, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.07395202046850076, "tokens": 120000, "cumulative_loss_tokens": 6000000, "grad_norm": 0.416015625, "lr": 3e-05, "finish_rate": 0.77, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.304, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.3, "frames": {"chat": 213}, "mem_gb": 16.09, "mem_gb_teacher": 16.09} |
| [eval step 50] sample: "To solve the given system of equations, we need to determine the values of \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\) such that each letter represents a non-zero digit. Let's break down the problem step-by" |
| checkpoint snapshot queued -> outputs/healed/keep50_warmup_fixed_s1224/step0050 |
| {"step": 51, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.05859705059945894, "tokens": 120000, "cumulative_loss_tokens": 6120000, "grad_norm": 0.34765625, "lr": 3e-05, "finish_rate": 0.815, "comp_len": 540.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.453, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.8, "frames": {"chat": 222}, "mem_gb": 16.0, "mem_gb_teacher": 16.0} |
| {"step": 52, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0539097297622667, "tokens": 120000, "cumulative_loss_tokens": 6240000, "grad_norm": 0.353515625, "lr": 3e-05, "finish_rate": 0.889, "comp_len": 510.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.439, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.1, "frames": {"chat": 235}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 53, "epoch": 0, "training_mode": "off-policy", "forward_topk_kl": 0.0594748610290233, "tokens": 120000, "cumulative_loss_tokens": 6360000, "grad_norm": 0.328125, "lr": 3e-05, "finish_rate": 0.798, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.417, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.8, "frames": {"chat": 208}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 54, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.045099887475029875, "tokens": 120000, "cumulative_loss_tokens": 6480000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.733, "comp_len": 628.3, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.413, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 80.2, "frames": {"chat": 191}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 55, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03569839329215077, "tokens": 120000, "cumulative_loss_tokens": 6600000, "grad_norm": 0.2451171875, "lr": 3e-05, "finish_rate": 0.845, "comp_len": 547.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.523, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.0, "frames": {"chat": 219}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 56, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04039377661425, "tokens": 120000, "cumulative_loss_tokens": 6720000, "grad_norm": 0.302734375, "lr": 3e-05, "finish_rate": 0.778, "comp_len": 579.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.459, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.0, "frames": {"chat": 207}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 57, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.055602129854184265, "tokens": 120000, "cumulative_loss_tokens": 6840000, "grad_norm": 0.330078125, "lr": 3e-05, "finish_rate": 0.755, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.449, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.0, "frames": {"chat": 208}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 58, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.035877808295966436, "tokens": 120000, "cumulative_loss_tokens": 6960000, "grad_norm": 0.279296875, "lr": 3e-05, "finish_rate": 0.799, "comp_len": 547.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.288, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.8, "frames": {"chat": 219}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 59, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03478414489501932, "tokens": 120000, "cumulative_loss_tokens": 7080000, "grad_norm": 0.2578125, "lr": 3e-05, "finish_rate": 0.915, "comp_len": 487.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.329, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 99.7, "frames": {"chat": 246}, "mem_gb": 15.92, "mem_gb_teacher": 15.92} |
| {"step": 60, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04739725781680706, "tokens": 120000, "cumulative_loss_tokens": 7200000, "grad_norm": 0.330078125, "lr": 3e-05, "finish_rate": 0.704, "comp_len": 582.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.437, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.5, "frames": {"chat": 206}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| [eval step 60] sample: 'To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to determine the values of these variables such that each letter represents a non-zero digit.\n\nThe equations a' |
| {"step": 61, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03820183755345643, "tokens": 120000, "cumulative_loss_tokens": 7320000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.876, "comp_len": 515.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.375, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.5, "frames": {"chat": 233}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 62, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04388709897000032, "tokens": 120000, "cumulative_loss_tokens": 7440000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.847, "comp_len": 524.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.363, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.8, "frames": {"chat": 229}, "mem_gb": 15.92, "mem_gb_teacher": 15.92} |
| {"step": 63, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.032643947703313705, "tokens": 120000, "cumulative_loss_tokens": 7560000, "grad_norm": 0.236328125, "lr": 3e-05, "finish_rate": 0.864, "comp_len": 508.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.285, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.4, "frames": {"chat": 236}, "mem_gb": 15.95, "mem_gb_teacher": 15.95} |
| {"step": 64, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04086883016227123, "tokens": 120000, "cumulative_loss_tokens": 7680000, "grad_norm": 0.2451171875, "lr": 3e-05, "finish_rate": 0.87, "comp_len": 502.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.37, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.3, "frames": {"chat": 239}, "mem_gb": 15.84, "mem_gb_teacher": 15.84} |
| {"step": 65, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03435394472128246, "tokens": 120000, "cumulative_loss_tokens": 7800000, "grad_norm": 0.2177734375, "lr": 3e-05, "finish_rate": 0.867, "comp_len": 497.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.349, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 99.1, "frames": {"chat": 241}, "mem_gb": 15.96, "mem_gb_teacher": 15.96} |
| {"step": 66, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.036759108127855385, "tokens": 120000, "cumulative_loss_tokens": 7920000, "grad_norm": 0.2294921875, "lr": 3e-05, "finish_rate": 0.863, "comp_len": 531.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.44, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.4, "frames": {"chat": 226}, "mem_gb": 15.92, "mem_gb_teacher": 15.92} |
| {"step": 67, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.031785604377323765, "tokens": 120000, "cumulative_loss_tokens": 8040000, "grad_norm": 0.224609375, "lr": 3e-05, "finish_rate": 0.893, "comp_len": 512.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.32, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.3, "frames": {"chat": 234}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 68, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03207274362700991, "tokens": 120000, "cumulative_loss_tokens": 8160000, "grad_norm": 0.22265625, "lr": 3e-05, "finish_rate": 0.914, "comp_len": 466.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.401, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 102.7, "frames": {"chat": 257}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 69, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04908341010484534, "tokens": 120000, "cumulative_loss_tokens": 8280000, "grad_norm": 0.2890625, "lr": 3e-05, "finish_rate": 0.76, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.523, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.3, "frames": {"chat": 208}, "mem_gb": 16.1, "mem_gb_teacher": 16.1} |
| {"step": 70, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.043880132612407516, "tokens": 120000, "cumulative_loss_tokens": 8400000, "grad_norm": 0.267578125, "lr": 3e-05, "finish_rate": 0.763, "comp_len": 568.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.522, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.2, "frames": {"chat": 211}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| [eval step 70] sample: "To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to ensure that each letter represents a non-zero digit. Let's break down the problem step-by-step and solve it" |
| {"step": 71, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04658220293604148, "tokens": 120000, "cumulative_loss_tokens": 8520000, "grad_norm": 0.302734375, "lr": 3e-05, "finish_rate": 0.806, "comp_len": 528.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.407, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.8, "frames": {"chat": 227}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 72, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04197514075435077, "tokens": 120000, "cumulative_loss_tokens": 8640000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.796, "comp_len": 568.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.434, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.1, "frames": {"chat": 211}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 73, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03433373855294194, "tokens": 120000, "cumulative_loss_tokens": 8760000, "grad_norm": 0.248046875, "lr": 3e-05, "finish_rate": 0.861, "comp_len": 504.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.403, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.0, "frames": {"chat": 238}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 74, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.035312130354298275, "tokens": 120000, "cumulative_loss_tokens": 8880000, "grad_norm": 0.2255859375, "lr": 3e-05, "finish_rate": 0.835, "comp_len": 506.3, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.517, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.2, "frames": {"chat": 237}, "mem_gb": 16.08, "mem_gb_teacher": 16.08} |
| {"step": 75, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04418459653495035, "tokens": 120000, "cumulative_loss_tokens": 9000000, "grad_norm": 0.265625, "lr": 3e-05, "finish_rate": 0.721, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.339, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.6, "frames": {"chat": 208}, "mem_gb": 16.08, "mem_gb_teacher": 16.08} |
| {"step": 76, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03462567985369048, "tokens": 120000, "cumulative_loss_tokens": 9120000, "grad_norm": 0.212890625, "lr": 3e-05, "finish_rate": 0.801, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.588, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.3, "frames": {"chat": 221}, "mem_gb": 16.17, "mem_gb_teacher": 16.17} |
| {"step": 77, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.0355419263230792, "tokens": 120000, "cumulative_loss_tokens": 9240000, "grad_norm": 0.244140625, "lr": 3e-05, "finish_rate": 0.853, "comp_len": 517.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.479, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.6, "frames": {"chat": 232}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 78, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03847126886160113, "tokens": 120000, "cumulative_loss_tokens": 9360000, "grad_norm": 0.25, "lr": 3e-05, "finish_rate": 0.764, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.43, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.7, "frames": {"chat": 208}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 79, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03187043257508582, "tokens": 120000, "cumulative_loss_tokens": 9480000, "grad_norm": 0.232421875, "lr": 3e-05, "finish_rate": 0.837, "comp_len": 528.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.356, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 93.2, "frames": {"chat": 227}, "mem_gb": 15.96, "mem_gb_teacher": 15.96} |
| {"step": 80, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03652716889477645, "tokens": 120000, "cumulative_loss_tokens": 9600000, "grad_norm": 0.251953125, "lr": 3e-05, "finish_rate": 0.824, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.387, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.2, "frames": {"chat": 221}, "mem_gb": 15.99, "mem_gb_teacher": 15.99} |
| [eval step 80] sample: 'To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to find the values of these variables such that each letter represents a non-zero digit.\n\nThe equations are:\n1' |
| {"step": 81, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03261745297779174, "tokens": 120000, "cumulative_loss_tokens": 9720000, "grad_norm": 0.2138671875, "lr": 3e-05, "finish_rate": 0.815, "comp_len": 517.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.457, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.5, "frames": {"chat": 232}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 82, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03819663266551991, "tokens": 120000, "cumulative_loss_tokens": 9840000, "grad_norm": 0.255859375, "lr": 3e-05, "finish_rate": 0.822, "comp_len": 547.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.423, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.2, "frames": {"chat": 219}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 83, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03785421463410991, "tokens": 120000, "cumulative_loss_tokens": 9960000, "grad_norm": 0.2265625, "lr": 3e-05, "finish_rate": 0.713, "comp_len": 615.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.399, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 82.7, "frames": {"chat": 195}, "mem_gb": 16.14, "mem_gb_teacher": 16.14} |
| {"step": 84, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03711687127229913, "tokens": 120000, "cumulative_loss_tokens": 10080000, "grad_norm": 0.244140625, "lr": 3e-05, "finish_rate": 0.833, "comp_len": 555.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.503, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.9, "frames": {"chat": 216}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 85, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.042023429202785095, "tokens": 120000, "cumulative_loss_tokens": 10200000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.788, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.37, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.7, "frames": {"chat": 208}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| {"step": 86, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03288566896258077, "tokens": 120000, "cumulative_loss_tokens": 10320000, "grad_norm": 0.2265625, "lr": 3e-05, "finish_rate": 0.919, "comp_len": 510.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.407, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.6, "frames": {"chat": 235}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| {"step": 87, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.033694713056855834, "tokens": 120000, "cumulative_loss_tokens": 10440000, "grad_norm": 0.2578125, "lr": 3e-05, "finish_rate": 0.853, "comp_len": 533.3, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.391, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.6, "frames": {"chat": 225}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 88, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04422600561644261, "tokens": 120000, "cumulative_loss_tokens": 10560000, "grad_norm": 0.2451171875, "lr": 3e-05, "finish_rate": 0.77, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.483, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.2, "frames": {"chat": 213}, "mem_gb": 16.13, "mem_gb_teacher": 16.13} |
| {"step": 89, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.029030231156169126, "tokens": 120000, "cumulative_loss_tokens": 10680000, "grad_norm": 0.2255859375, "lr": 3e-05, "finish_rate": 0.922, "comp_len": 466.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.372, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 104.0, "frames": {"chat": 257}, "mem_gb": 15.8, "mem_gb_teacher": 15.8} |
| {"step": 90, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.043001266495510934, "tokens": 120000, "cumulative_loss_tokens": 10800000, "grad_norm": 0.2578125, "lr": 3e-05, "finish_rate": 0.792, "comp_len": 566.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.497, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.1, "frames": {"chat": 212}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| [eval step 90] sample: 'To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to find the values of these variables such that each letter represents a non-zero digit.\n\nThe equations are:\n\\' |
| {"step": 91, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03456530287990657, "tokens": 120000, "cumulative_loss_tokens": 10920000, "grad_norm": 0.263671875, "lr": 3e-05, "finish_rate": 0.833, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.338, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.9, "frames": {"chat": 221}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 92, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.030914492412268495, "tokens": 120000, "cumulative_loss_tokens": 11040000, "grad_norm": 0.203125, "lr": 3e-05, "finish_rate": 0.868, "comp_len": 495.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.442, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.2, "frames": {"chat": 242}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 93, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.0329031198489829, "tokens": 120000, "cumulative_loss_tokens": 11160000, "grad_norm": 0.2216796875, "lr": 3e-05, "finish_rate": 0.836, "comp_len": 545.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.352, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.5, "frames": {"chat": 220}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 94, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.029550562638859263, "tokens": 120000, "cumulative_loss_tokens": 11280000, "grad_norm": 0.2197265625, "lr": 3e-05, "finish_rate": 0.896, "comp_len": 500.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.291, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.4, "frames": {"chat": 240}, "mem_gb": 15.9, "mem_gb_teacher": 15.9} |
| {"step": 95, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.0336006456746875, "tokens": 120000, "cumulative_loss_tokens": 11400000, "grad_norm": 0.23046875, "lr": 3e-05, "finish_rate": 0.728, "comp_len": 582.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.359, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.0, "frames": {"chat": 206}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 96, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04226834708025368, "tokens": 120000, "cumulative_loss_tokens": 11520000, "grad_norm": 0.255859375, "lr": 3e-05, "finish_rate": 0.867, "comp_len": 531.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.402, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.9, "frames": {"chat": 226}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 97, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.054896473065484314, "tokens": 120000, "cumulative_loss_tokens": 11640000, "grad_norm": 0.3046875, "lr": 3e-05, "finish_rate": 0.877, "comp_len": 491.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.382, "t_data_s": 0.1, "t_rollout_s": 0.0, "t_step_s": 99.8, "frames": {"chat": 244}, "mem_gb": 15.83, "mem_gb_teacher": 15.83} |
| {"step": 98, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.040829892701484884, "tokens": 120000, "cumulative_loss_tokens": 11760000, "grad_norm": 0.271484375, "lr": 3e-05, "finish_rate": 0.804, "comp_len": 535.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.406, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.6, "frames": {"chat": 224}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 99, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.031381215056039705, "tokens": 120000, "cumulative_loss_tokens": 11880000, "grad_norm": 0.240234375, "lr": 3e-05, "finish_rate": 0.923, "comp_len": 442.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.319, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 109.0, "frames": {"chat": 271}, "mem_gb": 15.78, "mem_gb_teacher": 15.78} |
| {"step": 100, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.036837965581729075, "tokens": 120000, "cumulative_loss_tokens": 12000000, "grad_norm": 0.265625, "lr": 3e-05, "finish_rate": 0.856, "comp_len": 508.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.362, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.5, "frames": {"chat": 236}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| [eval step 100] sample: "To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to ensure that each letter represents a non-zero digit. Let's break down the problem step-by-step and solve it" |
| checkpoint snapshot queued -> outputs/healed/keep50_warmup_fixed_s1224/step0100 |
| {"step": 101, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03681235469069021, "tokens": 120000, "cumulative_loss_tokens": 12120000, "grad_norm": 0.244140625, "lr": 3e-05, "finish_rate": 0.841, "comp_len": 517.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.306, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.0, "frames": {"chat": 232}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| {"step": 102, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03435125927827321, "tokens": 120000, "cumulative_loss_tokens": 12240000, "grad_norm": 0.251953125, "lr": 3e-05, "finish_rate": 0.79, "comp_len": 571.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.392, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.6, "frames": {"chat": 210}, "mem_gb": 15.98, "mem_gb_teacher": 15.98} |
| {"step": 103, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.033812837561887375, "tokens": 120000, "cumulative_loss_tokens": 12360000, "grad_norm": 0.275390625, "lr": 3e-05, "finish_rate": 0.811, "comp_len": 553.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.431, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.9, "frames": {"chat": 217}, "mem_gb": 15.95, "mem_gb_teacher": 15.95} |
| {"step": 104, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.03568466838332825, "tokens": 120000, "cumulative_loss_tokens": 12480000, "grad_norm": 0.25390625, "lr": 3e-05, "finish_rate": 0.839, "comp_len": 535.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.38, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 92.7, "frames": {"chat": 224}, "mem_gb": 16.07, "mem_gb_teacher": 16.07} |
| {"step": 105, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.04380085541328881, "tokens": 120000, "cumulative_loss_tokens": 12600000, "grad_norm": 0.287109375, "lr": 3e-05, "finish_rate": 0.749, "comp_len": 591.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.481, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.5, "frames": {"chat": 203}, "mem_gb": 15.92, "mem_gb_teacher": 15.92} |
| {"step": 106, "epoch": 1, "training_mode": "off-policy", "forward_topk_kl": 0.0331037232719129, "tokens": 120000, "cumulative_loss_tokens": 12720000, "grad_norm": 0.2197265625, "lr": 3e-05, "finish_rate": 0.887, "comp_len": 502.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.326, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 97.2, "frames": {"chat": 239}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 107, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.022136363052297384, "tokens": 120000, "cumulative_loss_tokens": 12840000, "grad_norm": 0.1875, "lr": 3e-05, "finish_rate": 0.902, "comp_len": 472.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.301, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 102.8, "frames": {"chat": 254}, "mem_gb": 15.93, "mem_gb_teacher": 15.93} |
| {"step": 108, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02345696447007358, "tokens": 120000, "cumulative_loss_tokens": 12960000, "grad_norm": 0.21875, "lr": 3e-05, "finish_rate": 0.876, "comp_len": 497.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.421, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.5, "frames": {"chat": 241}, "mem_gb": 16.03, "mem_gb_teacher": 16.03} |
| {"step": 109, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.03220615579979494, "tokens": 120000, "cumulative_loss_tokens": 13080000, "grad_norm": 0.2294921875, "lr": 3e-05, "finish_rate": 0.746, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.419, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.1, "frames": {"chat": 213}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 110, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.028369275209781095, "tokens": 120000, "cumulative_loss_tokens": 13200000, "grad_norm": 0.205078125, "lr": 3e-05, "finish_rate": 0.864, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.641, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.4, "frames": {"chat": 221}, "mem_gb": 16.1, "mem_gb_teacher": 16.1} |
| [eval step 110] sample: "To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to ensure that each letter represents a non-zero digit. Let's break down the problem step-by-step and solve it" |
| {"step": 111, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.030552633555675855, "tokens": 120000, "cumulative_loss_tokens": 13320000, "grad_norm": 0.201171875, "lr": 3e-05, "finish_rate": 0.745, "comp_len": 612.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.334, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 83.1, "frames": {"chat": 196}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 112, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02257079966662762, "tokens": 120000, "cumulative_loss_tokens": 13440000, "grad_norm": 0.1689453125, "lr": 3e-05, "finish_rate": 0.926, "comp_len": 444.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.427, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 108.7, "frames": {"chat": 270}, "mem_gb": 15.86, "mem_gb_teacher": 15.86} |
| {"step": 113, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.024793952927171875, "tokens": 120000, "cumulative_loss_tokens": 13560000, "grad_norm": 0.2041015625, "lr": 3e-05, "finish_rate": 0.815, "comp_len": 555.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.327, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.5, "frames": {"chat": 216}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 114, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02779970950682958, "tokens": 120000, "cumulative_loss_tokens": 13680000, "grad_norm": 0.1923828125, "lr": 3e-05, "finish_rate": 0.775, "comp_len": 600.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.314, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 83.9, "frames": {"chat": 200}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 115, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.025435146312182768, "tokens": 120000, "cumulative_loss_tokens": 13800000, "grad_norm": 0.19921875, "lr": 3e-05, "finish_rate": 0.767, "comp_len": 582.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.406, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.4, "frames": {"chat": 206}, "mem_gb": 15.96, "mem_gb_teacher": 15.96} |
| {"step": 116, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.021958818318624982, "tokens": 120000, "cumulative_loss_tokens": 13920000, "grad_norm": 0.1650390625, "lr": 3e-05, "finish_rate": 0.902, "comp_len": 512.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.5, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 94.8, "frames": {"chat": 234}, "mem_gb": 15.99, "mem_gb_teacher": 15.99} |
| {"step": 117, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.024690080278979926, "tokens": 120000, "cumulative_loss_tokens": 14040000, "grad_norm": 0.171875, "lr": 3e-05, "finish_rate": 0.823, "comp_len": 558.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.325, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.4, "frames": {"chat": 215}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 118, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02221767870101612, "tokens": 120000, "cumulative_loss_tokens": 14160000, "grad_norm": 0.2021484375, "lr": 3e-05, "finish_rate": 0.922, "comp_len": 470.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.447, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 101.5, "frames": {"chat": 255}, "mem_gb": 15.99, "mem_gb_teacher": 15.99} |
| {"step": 119, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.021592201069470806, "tokens": 120000, "cumulative_loss_tokens": 14280000, "grad_norm": 0.1708984375, "lr": 3e-05, "finish_rate": 0.892, "comp_len": 480.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.377, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 101.8, "frames": {"chat": 250}, "mem_gb": 15.87, "mem_gb_teacher": 15.87} |
| {"step": 120, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.022036931684900386, "tokens": 120000, "cumulative_loss_tokens": 14400000, "grad_norm": 0.1826171875, "lr": 3e-05, "finish_rate": 0.884, "comp_len": 495.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.525, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 97.8, "frames": {"chat": 242}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| [eval step 120] sample: 'To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to find the values of these variables such that each letter represents a non-zero digit.\n\nThe equations are:\n\\' |
| {"step": 121, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.027385966173838823, "tokens": 120000, "cumulative_loss_tokens": 14520000, "grad_norm": 0.177734375, "lr": 3e-05, "finish_rate": 0.729, "comp_len": 603.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.517, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 85.1, "frames": {"chat": 199}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 122, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.037581453007894255, "tokens": 120000, "cumulative_loss_tokens": 14640000, "grad_norm": 0.2138671875, "lr": 3e-05, "finish_rate": 0.784, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.386, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.8, "frames": {"chat": 208}, "mem_gb": 16.08, "mem_gb_teacher": 16.08} |
| {"step": 123, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02500725610067602, "tokens": 120000, "cumulative_loss_tokens": 14760000, "grad_norm": 0.212890625, "lr": 3e-05, "finish_rate": 0.764, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.304, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 86.8, "frames": {"chat": 208}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 124, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.029994825281358013, "tokens": 120000, "cumulative_loss_tokens": 14880000, "grad_norm": 0.2041015625, "lr": 3e-05, "finish_rate": 0.732, "comp_len": 574.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.473, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.1, "frames": {"chat": 209}, "mem_gb": 16.17, "mem_gb_teacher": 16.17} |
| {"step": 125, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.021332296466493667, "tokens": 120000, "cumulative_loss_tokens": 15000000, "grad_norm": 0.1728515625, "lr": 3e-05, "finish_rate": 0.855, "comp_len": 510.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.321, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 97.2, "frames": {"chat": 235}, "mem_gb": 16.0, "mem_gb_teacher": 16.0} |
| {"step": 126, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02423879373523717, "tokens": 120000, "cumulative_loss_tokens": 15120000, "grad_norm": 0.2138671875, "lr": 3e-05, "finish_rate": 0.74, "comp_len": 588.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.503, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 85.5, "frames": {"chat": 204}, "mem_gb": 16.0, "mem_gb_teacher": 16.0} |
| {"step": 127, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.03143483543585365, "tokens": 120000, "cumulative_loss_tokens": 15240000, "grad_norm": 0.24609375, "lr": 3e-05, "finish_rate": 0.745, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.498, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.5, "frames": {"chat": 208}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 128, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.022677327596287555, "tokens": 120000, "cumulative_loss_tokens": 15360000, "grad_norm": 0.1748046875, "lr": 3e-05, "finish_rate": 0.825, "comp_len": 500.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.456, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.6, "frames": {"chat": 240}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 129, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02317165856284555, "tokens": 120000, "cumulative_loss_tokens": 15480000, "grad_norm": 0.18359375, "lr": 3e-05, "finish_rate": 0.89, "comp_len": 487.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.425, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 99.3, "frames": {"chat": 246}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 130, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02553549518882452, "tokens": 120000, "cumulative_loss_tokens": 15600000, "grad_norm": 0.16796875, "lr": 3e-05, "finish_rate": 0.909, "comp_len": 493.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.299, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 98.1, "frames": {"chat": 243}, "mem_gb": 15.86, "mem_gb_teacher": 15.86} |
| [eval step 130] sample: "To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to ensure that each letter represents a non-zero digit. Let's break down the problem step-by-step and solve it" |
| {"step": 131, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.03025520235665608, "tokens": 120000, "cumulative_loss_tokens": 15720000, "grad_norm": 0.193359375, "lr": 3e-05, "finish_rate": 0.745, "comp_len": 576.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.449, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.0, "frames": {"chat": 208}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 132, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.033440696114464666, "tokens": 120000, "cumulative_loss_tokens": 15840000, "grad_norm": 0.20703125, "lr": 3e-05, "finish_rate": 0.817, "comp_len": 547.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.352, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.4, "frames": {"chat": 219}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 133, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.030961406842339785, "tokens": 120000, "cumulative_loss_tokens": 15960000, "grad_norm": 0.205078125, "lr": 3e-05, "finish_rate": 0.782, "comp_len": 568.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.447, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 89.4, "frames": {"chat": 211}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 134, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.030365196120288845, "tokens": 120000, "cumulative_loss_tokens": 16080000, "grad_norm": 0.25, "lr": 3e-05, "finish_rate": 0.862, "comp_len": 517.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.283, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.3, "frames": {"chat": 232}, "mem_gb": 16.02, "mem_gb_teacher": 16.02} |
| {"step": 135, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.03637079153128434, "tokens": 120000, "cumulative_loss_tokens": 16200000, "grad_norm": 0.201171875, "lr": 3e-05, "finish_rate": 0.804, "comp_len": 560.7, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.403, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 90.4, "frames": {"chat": 214}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| {"step": 136, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.024788761290698312, "tokens": 120000, "cumulative_loss_tokens": 16320000, "grad_norm": 0.173828125, "lr": 3e-05, "finish_rate": 0.819, "comp_len": 531.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.395, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 94.4, "frames": {"chat": 226}, "mem_gb": 15.94, "mem_gb_teacher": 15.94} |
| {"step": 137, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02415202263732596, "tokens": 120000, "cumulative_loss_tokens": 16440000, "grad_norm": 0.171875, "lr": 3e-05, "finish_rate": 0.819, "comp_len": 571.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.404, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.7, "frames": {"chat": 210}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 138, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.023906889412474507, "tokens": 120000, "cumulative_loss_tokens": 16560000, "grad_norm": 0.1884765625, "lr": 3e-05, "finish_rate": 0.858, "comp_len": 550.5, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.436, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.6, "frames": {"chat": 218}, "mem_gb": 15.88, "mem_gb_teacher": 15.88} |
| {"step": 139, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02281373731412459, "tokens": 120000, "cumulative_loss_tokens": 16680000, "grad_norm": 0.185546875, "lr": 3e-05, "finish_rate": 0.858, "comp_len": 515.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.354, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 96.0, "frames": {"chat": 233}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 140, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.029389094779423128, "tokens": 120000, "cumulative_loss_tokens": 16800000, "grad_norm": 0.1962890625, "lr": 3e-05, "finish_rate": 0.786, "comp_len": 558.1, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.221, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.1, "frames": {"chat": 215}, "mem_gb": 16.05, "mem_gb_teacher": 16.05} |
| [eval step 140] sample: 'To solve the given system of equations with the constraint that each letter represents a non-zero digit, we need to find the values of \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\) that satisfy all the equati' |
| {"step": 141, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02496639485837271, "tokens": 120000, "cumulative_loss_tokens": 16920000, "grad_norm": 0.1826171875, "lr": 3e-05, "finish_rate": 0.845, "comp_len": 515.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.428, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.0, "frames": {"chat": 233}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 142, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.024896287856064736, "tokens": 120000, "cumulative_loss_tokens": 17040000, "grad_norm": 0.2138671875, "lr": 3e-05, "finish_rate": 0.766, "comp_len": 574.2, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.353, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.5, "frames": {"chat": 209}, "mem_gb": 15.99, "mem_gb_teacher": 15.99} |
| {"step": 143, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.0202058710468933, "tokens": 120000, "cumulative_loss_tokens": 17160000, "grad_norm": 0.166015625, "lr": 3e-05, "finish_rate": 0.908, "comp_len": 458.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.308, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 105.8, "frames": {"chat": 262}, "mem_gb": 15.92, "mem_gb_teacher": 15.92} |
| {"step": 144, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.0228134301078273, "tokens": 120000, "cumulative_loss_tokens": 17280000, "grad_norm": 0.171875, "lr": 3e-05, "finish_rate": 0.9, "comp_len": 481.9, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.241, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 101.0, "frames": {"chat": 249}, "mem_gb": 16.01, "mem_gb_teacher": 16.01} |
| {"step": 145, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.030403959429240787, "tokens": 120000, "cumulative_loss_tokens": 17400000, "grad_norm": 0.18359375, "lr": 3e-05, "finish_rate": 0.819, "comp_len": 528.6, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.364, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 94.0, "frames": {"chat": 227}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 146, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.022969909678919553, "tokens": 120000, "cumulative_loss_tokens": 17520000, "grad_norm": 0.1796875, "lr": 3e-05, "finish_rate": 0.814, "comp_len": 543.0, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.356, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 91.2, "frames": {"chat": 221}, "mem_gb": 16.04, "mem_gb_teacher": 16.04} |
| {"step": 147, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.02228280872052225, "tokens": 120000, "cumulative_loss_tokens": 17640000, "grad_norm": 0.171875, "lr": 3e-05, "finish_rate": 0.859, "comp_len": 512.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.473, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.0, "frames": {"chat": 234}, "mem_gb": 16.06, "mem_gb_teacher": 16.06} |
| {"step": 148, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.023852225604599032, "tokens": 120000, "cumulative_loss_tokens": 17760000, "grad_norm": 0.212890625, "lr": 3e-05, "finish_rate": 0.817, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.29, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 87.8, "frames": {"chat": 213}, "mem_gb": 16.0, "mem_gb_teacher": 16.0} |
| {"step": 149, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.022865247969034438, "tokens": 120000, "cumulative_loss_tokens": 17880000, "grad_norm": 0.1748046875, "lr": 3e-05, "finish_rate": 0.836, "comp_len": 563.4, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.455, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 88.1, "frames": {"chat": 213}, "mem_gb": 15.94, "mem_gb_teacher": 15.94} |
| {"step": 150, "epoch": 2, "training_mode": "off-policy", "forward_topk_kl": 0.021284172641811892, "tokens": 120000, "cumulative_loss_tokens": 18000000, "grad_norm": 0.1669921875, "lr": 3e-05, "finish_rate": 0.906, "comp_len": 512.8, "dropped_truncated": 0, "gold_loss": null, "gold_lambda": null, "rep_ratio": 2.392, "t_data_s": 0.0, "t_rollout_s": 0.0, "t_step_s": 95.1, "frames": {"chat": 234}, "mem_gb": 15.97, "mem_gb_teacher": 15.97} |
| [eval step 150] sample: "To solve the given system of equations for \\(a\\), \\(b\\), \\(k\\), \\(m\\), and \\(p\\), we need to find the values of these variables such that each letter represents a non-zero digit.\n\nLet's break down the" |
| checkpoint snapshot queued -> outputs/healed/keep50_warmup_fixed_s1224/step0150 |
| wandb: updating run metadata |
| wandb: uploading output.log; uploading wandb-summary.json; uploading config.yaml |
| wandb: |
| wandb: Run history: |
| wandb: comp_len ▆▃▆▃▄▄▆▃▃▅▆▅▄█▅▄▄▆▅▆▅▅▂▃▄▃▆▅▇▃▁▇▆▄▂▃▅▅▂▅ |
| wandb: cumulative_loss_tokens ▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▄▄▄▄▄▅▅▅▆▆▆▆▇▇▇▇▇▇▇▇▇████ |
| wandb: dropped_truncated ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ |
| wandb: epoch ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅▅▅▅▅▅▅▅▅▅▅▅▅██████████ |
| wandb: finish_rate ▅▄▂▃▆▆▅▃▇▅█▆▆▇█▂▄▄█▆▃▅▆▄▅▃▂▇▇▁▂▇█▂▆▄▆▅█▄ |
| wandb: forward_topk_kl █▆▄▄▃▃▄▂▂▃▂▂▂▂▂▂▂▁▂▂▁▁▂▁▁▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁ |
| wandb: grad_norm █▂▂▂▂▂▂▂▂▁▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ |
| wandb: lr ▁███████████████████████████████████████ |
| wandb: mem_gb ▃▆▆▄▂▄▇▆▇▃▆▆▆▅▇▆▅▅▆█▆█▆▅▆▁▄▆▆▅▃▆▅▃▆▆▄▅▆▅ |
| wandb: mem_gb_teacher ▅▄▆▇▆▂▆▆▆▇▆▅▃█▆▆▆▅▆▃▆▄▂▃▆█▄▄█▁▆▇▂▆▅▅▆▆▆▃ |
| wandb: +6 ... |
| wandb: |
| wandb: Run summary: |
| wandb: comp_len 512.8 |
| wandb: cumulative_loss_tokens 18000000 |
| wandb: dropped_truncated 0 |
| wandb: epoch 2 |
| wandb: finish_rate 0.906 |
| wandb: forward_topk_kl 0.02128 |
| wandb: grad_norm 0.16699 |
| wandb: lr 3e-05 |
| wandb: mem_gb 15.97 |
| wandb: mem_gb_teacher 15.97 |
| wandb: +7 ... |
| wandb: |
| wandb: 🚀 View run warmup-fixed-keep50-s1224 at: https://wandb.ai/hbfreed/glean-heal/runs/sep45w2j |
| wandb: ⭐️ View project at: https://wandb.ai/hbfreed/glean-heal |
| wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) |
| wandb: Find logs at: outputs/healed/keep50_warmup_fixed_s1224/wandb/run-20260802_025702-sep45w2j/logs |
| |
