Add L12 sampled pressure ablation

docs/experiment_summary_report.md

@@ -1,12 +1,10 @@
 # Experiment Summary Report
 
-Date: 2026-05-27
+Date: 2026-05-28
 
-This report summarizes completed experiments only. The active run
-`runs/formula_l10_interpolation/locked_stream/20260527-153920` is excluded
-because it has no completed `summary.json` or `RESULT_SUMMARY.md` yet.
-
-No new experiments were run for this report.
+This report summarizes completed experiments only. It includes the model-size
+interpolation, update-pressure, and sampled-pressure ablation runs completed
+after the first report draft.
 
 ## Executive Summary
 
@@ -33,8 +31,8 @@ The strongest completed evidence is:
   finished at `4.6094 +/- 0.0056`, better than static `0.08` at
   `4.6257 +/- 0.0007`, but its mean trajectory was not best.
 
-The pattern is consistent but still needs the active L10 interpolation run to
-test whether the effect varies smoothly with model size.
+The pattern is consistent across L8/L10/L12/L14/L16 and now supports a smooth
+model-size interpretation, with L8 remaining the weakest boundary case.
 
 ## Completed Run Inventory
 
@@ -50,7 +48,11 @@ test whether the effect varies smoothly with model size.
 | `l12_followup_085421` | complete | L12 seeds 2 and 3 follow-up for common conditions |
 | `l8_boundary_104407` | complete | L8 3-seed boundary test |
 | `l16_exact_formula_123806` | complete | L16 exact formula-vs-static confirmation |
-| `l10_interpolation_153920` | active, excluded | L10 interpolation run |
+| `l10_interpolation_153920` | complete | L10 interpolation run |
+| `l14_interpolation_182113` | complete | L14 interpolation run |
+| `l12_stage_steps_500_231804` | complete | L12 low-update pressure validation |
+| `l12_stage_steps_2000_004033` | complete | L12 high-update pressure validation |
+| `l12_sample_pressure_ablation_053842` | complete | L12 sampled-pressure coefficient ablation |
 
 Full artifact paths are preserved in the run directories; this table uses short
 IDs to keep the PDF readable.
@@ -129,9 +131,9 @@ The actual tested formula anchors were:
 | L12 | 17.37M | `0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02` |
 | L16 | 31.46M | `0.34 -> 0.29 -> 0.22 -> 0.13 -> 0.03` |
 
-The active L10 interpolation run is the most direct check of this formula. If it
-lands between L8 and L12/L16, the capacity-pressure explanation becomes more
-credible. If not, the formula may be overfit to the completed anchor models.
+The completed L10 and L14 interpolation runs are direct checks of this formula.
+Both landed between the prior anchor models, making the capacity-pressure
+explanation more credible than a small number of hand-selected anchors.
 
 ## L16 Schedule Development
 
@@ -241,10 +243,59 @@ Not supported:
 - High-initial dropout decay explains the original streaming improvement.
 - The 8.39M legacy setup has a robust nonzero static optimum above zero.
 
+## Formula Validation Addendum
+
+Additional runs completed after the first report strengthen the empirical
+formula. L10 and L14 interpolation runs landed between the L8/L12/L16 results,
+and L12 update-pressure runs showed that the sampled-token term is directionally
+important.
+
+### Model-Size Interpolation
+
+| Model | Params | Formula path | Formula final val | Best static final val | Paired delta vs best static |
+|---|---:|---|---:|---:|---:|
+| L8 | 8.39M | `0.252 -> 0.206 -> 0.129 -> 0.038 -> 0.020` | `4.6094 +/- 0.0056` | `4.6242` | `-0.0102, -0.0160, -0.0182` |
+| L10 | 12.31M | `0.278 -> 0.232 -> 0.154 -> 0.064 -> 0.020` | `4.5306 +/- 0.0094` | `4.5580` | `-0.0288, -0.0188, -0.0345` |
+| L12 | 17.37M | `0.300 -> 0.260 -> 0.180 -> 0.090 -> 0.020` | `4.4812 +/- 0.0062` | `4.5183` | `-0.0364, -0.0308, -0.0439` |
+| L14 | 23.70M | `0.322 -> 0.276 -> 0.198 -> 0.108 -> 0.020` | `4.4384 +/- 0.0087` | `4.4736` | `-0.0294, -0.0269, -0.0429` |
+| L16 | 31.46M | `0.341 -> 0.294 -> 0.217 -> 0.127 -> 0.030` | `4.4059 +/- 0.0046` | `4.4459` | `-0.0411, -0.0512, -0.0279` |
+
+### L12 Update-Pressure Sweep
+
+The formula includes `log10(cumulative_sampled_tokens / unique_tokens)`.
+Changing `stage_steps` changes this ratio while keeping the model and stream
+prefixes fixed.
+
+| Stage steps | Formula path | Mean trajectory val | Formula final val | Best static final val | Paired delta vs best static |
+|---:|---|---:|---:|---:|---:|
+| 500 | `0.226 -> 0.180 -> 0.102 -> 0.020 -> 0.020` | `5.1581` | `4.7138 +/- 0.0080` | `4.7321` | `-0.0152, -0.0147, -0.0249` |
+| 1000 | `0.300 -> 0.260 -> 0.180 -> 0.090 -> 0.020` | `4.9226` | `4.4812 +/- 0.0062` | `4.5183` | `-0.0364, -0.0308, -0.0439` |
+| 2000 | `0.376 -> 0.330 -> 0.252 -> 0.162 -> 0.065` | `4.7841` | `4.3089 +/- 0.0116` | `4.3513` | `-0.0453, -0.0321, -0.0489` |
+
+The low-update regime still wins final loss, but not mean trajectory. The
+high-update regime wins both final and trajectory, matching the claim that
+repeated sampling from the same prefix increases the needed dropout.
+
+### Sampled-Pressure Coefficient Ablation
+
+These L12 runs keep the model, stream caps, and training budget fixed while
+scaling only the sampled-pressure coefficient.
+
+| Condition | Pressure coefficient | Path | Mean trajectory val | Final val | Final std | Final gap | N |
+|---|---:|---|---:|---:|---:|---:|---:|
+| `no_sample_pressure_l12` | 0x | `0.074 -> 0.027 -> 0.020 -> 0.020 -> 0.020` | `5.0282` | `4.5468` | `0.0011` | `0.3482` | 3 |
+| `half_sample_pressure_l12` | 0.5x | `0.187 -> 0.141 -> 0.079 -> 0.020 -> 0.020` | `4.9260` | `4.5055` | `0.0046` | `0.3272` | 3 |
+| `pressure_formula_floor02` | 1.0x | `0.300 -> 0.260 -> 0.180 -> 0.090 -> 0.020` | `4.9226` | `4.4812` | `0.0062` | `0.2825` | 3 |
+| `high_sample_pressure_l12` | 1.5x | `0.415 -> 0.368 -> 0.275 -> 0.163 -> 0.041` | `4.9739` | `4.4959` | `0.0025` | `0.2418` | 3 |
+
+The 1.0x fitted coefficient is best on final validation loss. The 1.5x variant
+has the smallest final gap but worse validation, so the objective is not simply
+to minimize the train-validation gap. The coefficient appears to be near the
+useful range for L12 under the current stream protocol.
+
 Open:
 
-- Whether the improvement varies smoothly with model size between L8, L12, and
-  L16. The active L10 interpolation run is designed to probe this.
+- Whether the same sampled-pressure coefficient ablation holds on L8 and L16.
 - Whether final-loss wins are accompanied by consistently better integrated
   trajectory loss. L8 shows a final-loss win without the best trajectory mean.
 - Whether the `0.02` floor is necessary. L12 seed 1 showed raw-to-zero and
@@ -258,7 +309,6 @@ The best framing right now is not "dropout decay works." It is:
 > with stream size. The schedule must be anchored to measured static optima; high
 > arbitrary initial dropout fails.
 
-The active L10 result should be used as the next decision point. If L10 lands
-between L8 and L12/L16, the model-size interpolation story becomes much more
-credible. If it does not, the formula may be exploiting per-model noise or a
-small number of hand-selected anchors.
+The next decision point should test whether the same formula transfers outside
+the current corpus/protocol and whether the sampled-pressure coefficient
+ablation reproduces on L8 and L16.

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/RESULT_SUMMARY.md

@@ -0,0 +1,56 @@
+# Locked Streaming Dropout Summary
+
+Run directory: `runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842`
+
+Model: `L12_H8_D320` causal Transformer, 17,367,040 parameters, 12 layers, 8 heads, 320 embedding dim.
+Training per stage: 1,000 steps. Sampled tokens are cumulative in each stage row. Seeds present: 1, 2, 3.
+
+## Condition Ranking
+
+| Condition | Kind | Final dropout | Mean trajectory val loss | Final val loss | Final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---|
+| `half_sample_pressure_l12` | anchor_decay | 0.02 | 4.9260 | 4.5055 | 0.3272 | 0.19 -> 0.14 -> 0.08 -> 0.02 -> 0.02 |
+| `high_sample_pressure_l12` | anchor_decay | 0.04 | 4.9739 | 4.4959 | 0.2418 | 0.41 -> 0.37 -> 0.28 -> 0.16 -> 0.04 |
+| `no_sample_pressure_l12` | anchor_decay | 0.02 | 5.0282 | 4.5468 | 0.3482 | 0.07 -> 0.03 -> 0.02 -> 0.02 -> 0.02 |
+
+## Stage Trajectory
+
+### Stage 0: 250,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `half_sample_pressure_l12` | 0.19 | 5.4776 | 4.4933 | 0.9842 | 3 |
+| `no_sample_pressure_l12` | 0.07 | 5.5299 | 4.2264 | 1.3035 | 3 |
+| `high_sample_pressure_l12` | 0.41 | 5.6181 | 4.9182 | 0.6999 | 3 |
+
+### Stage 1: 500,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `half_sample_pressure_l12` | 0.14 | 5.1221 | 4.1207 | 1.0014 | 3 |
+| `high_sample_pressure_l12` | 0.37 | 5.1838 | 4.5413 | 0.6425 | 3 |
+| `no_sample_pressure_l12` | 0.03 | 5.3265 | 3.7633 | 1.5632 | 3 |
+
+### Stage 2: 1,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `half_sample_pressure_l12` | 0.08 | 4.8600 | 4.0818 | 0.7783 | 3 |
+| `high_sample_pressure_l12` | 0.28 | 4.8996 | 4.4004 | 0.4992 | 3 |
+| `no_sample_pressure_l12` | 0.02 | 5.0044 | 3.9008 | 1.1037 | 3 |
+
+### Stage 3: 2,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `half_sample_pressure_l12` | 0.02 | 4.6647 | 4.1221 | 0.5426 | 3 |
+| `high_sample_pressure_l12` | 0.16 | 4.6723 | 4.3334 | 0.3390 | 3 |
+| `no_sample_pressure_l12` | 0.02 | 4.7335 | 4.0840 | 0.6495 | 3 |
+
+### Stage 4: 4,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `high_sample_pressure_l12` | 0.04 | 4.4959 | 4.2541 | 0.2418 | 3 |
+| `half_sample_pressure_l12` | 0.02 | 4.5055 | 4.1782 | 0.3272 | 3 |
+| `no_sample_pressure_l12` | 0.02 | 4.5468 | 4.1986 | 0.3482 | 3 |

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/config.json

@@ -0,0 +1,182 @@
+{
+  "args": {
+    "mode": "locked_stream",
+    "corpus": null,
+    "corpus_glob": "/Users/mandeepsidhu/Desktop/code/nanochat/.nanochat-cache/base_data_climbmix/shard_*.parquet",
+    "text_column": "text",
+    "output_dir": "runs/formula_l12_sample_pressure_ablation",
+    "resume_from": null,
+    "cache_dir": ".cache/dropout_decay",
+    "models": [
+      "L12_H8_D320=12x8x320"
+    ],
+    "seeds": [
+      1,
+      2,
+      3
+    ],
+    "token_limits": [
+      5000000
+    ],
+    "stream_token_caps": [
+      250000,
+      500000,
+      1000000,
+      2000000,
+      4000000
+    ],
+    "val_tokens": 500000,
+    "allow_short_corpus": false,
+    "force_retokenize": false,
+    "vocab_size": 4096,
+    "tokenizer_train_chars": 10000000,
+    "block_size": 128,
+    "batch_size": 16,
+    "steps": 2000,
+    "stage_steps": 1000,
+    "dropout_rates": [],
+    "decays": [],
+    "anchor_decays": [
+      {
+        "name": "no_sample_pressure_l12",
+        "kind": "anchor_decay",
+        "initial": 0.074,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.074
+          ],
+          [
+            500000,
+            0.027
+          ],
+          [
+            1000000,
+            0.02
+          ],
+          [
+            2000000,
+            0.02
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "half_sample_pressure_l12",
+        "kind": "anchor_decay",
+        "initial": 0.187,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.187
+          ],
+          [
+            500000,
+            0.141
+          ],
+          [
+            1000000,
+            0.079
+          ],
+          [
+            2000000,
+            0.02
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "high_sample_pressure_l12",
+        "kind": "anchor_decay",
+        "initial": 0.415,
+        "final": 0.041,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.415
+          ],
+          [
+            500000,
+            0.368
+          ],
+          [
+            1000000,
+            0.275
+          ],
+          [
+            2000000,
+            0.163
+          ],
+          [
+            4000000,
+            0.041
+          ]
+        ]
+      }
+    ],
+    "decay_tokens": null,
+    "eval_batches": 64,
+    "train_eval_batches": 32,
+    "trace_eval_batches": 8,
+    "eval_every": 0,
+    "log_every": 500,
+    "lr": 0.0003,
+    "weight_decay": 0.1,
+    "grad_clip": 1.0,
+    "plateau_delta": 0.01,
+    "target_min_dropout": 0.1,
+    "min_nonzero_margin": 0.01,
+    "min_high_dropout_margin": 0.03,
+    "screen_early_stop": false,
+    "screen_prune_patience": 3,
+    "screen_prune_min_delta": 0.01
+  },
+  "mode": "locked_stream",
+  "seeds": [
+    1,
+    2,
+    3
+  ],
+  "models": [
+    {
+      "model_name": "L12_H8_D320",
+      "n_layer": 12,
+      "n_head": 8,
+      "n_embd": 320
+    }
+  ],
+  "device": "mps",
+  "torch": "2.9.1",
+  "python": "3.10.20 (main, May 10 2026, 19:20:50) [Clang 22.1.3 ]",
+  "mps_available": true,
+  "attribution": "Derived from Andrej Karpathy's nanochat project (https://github.com/karpathy/nanochat), MIT License, Copyright (c) 2025 Andrej Karpathy.",
+  "tokenizer_path": ".cache/dropout_decay/tokenizer-v4096.json",
+  "encoded_path": ".cache/dropout_decay/tokens-v4096-uint16.npy",
+  "train_tokens": 5000970,
+  "val_tokens": 500000,
+  "effective_token_limits": [
+    5000000
+  ],
+  "effective_stream_token_caps": [
+    250000,
+    500000,
+    1000000,
+    2000000,
+    4000000
+  ],
+  "resume_from": null
+}

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/metrics.jsonl

@@ -0,0 +1,45 @@
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.074, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 66.09603786468506, "eval_loss": 5.534066058695316, "generalization_gap": 1.2835443541407585, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.250521704554558, "train_loss_last": 4.3242878913879395, "val_eval_loss": 5.534066058695316}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.026999999999999996, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 66.89631986618042, "eval_loss": 5.325545907020569, "generalization_gap": 1.5631037950515747, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.762442111968994, "train_loss_last": 4.084026336669922, "val_eval_loss": 5.325545907020569}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 72.65545415878296, "eval_loss": 5.001262731850147, "generalization_gap": 1.100004956126213, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.901257775723934, "train_loss_last": 3.7913968563079834, "val_eval_loss": 5.001262731850147}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.42098212242126, "eval_loss": 4.742752246558666, "generalization_gap": 0.6534350290894508, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.089317217469215, "train_loss_last": 4.254018306732178, "val_eval_loss": 4.742752246558666}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 90.97651791572571, "eval_loss": 4.5479796230793, "generalization_gap": 0.3460318595170975, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.2019477635622025, "train_loss_last": 4.2256083488464355, "val_eval_loss": 4.5479796230793}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.074, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 87.11983299255371, "eval_loss": 5.542020536959171, "generalization_gap": 1.3205124363303185, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.221508100628853, "train_loss_last": 4.35040283203125, "val_eval_loss": 5.542020536959171}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.026999999999999996, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 85.65824007987976, "eval_loss": 5.314211867749691, "generalization_gap": 1.568748876452446, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.745462991297245, "train_loss_last": 3.9610917568206787, "val_eval_loss": 5.314211867749691}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 85.30307912826538, "eval_loss": 5.0040023624897, "generalization_gap": 1.0687210112810135, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.935281351208687, "train_loss_last": 3.683029890060425, "val_eval_loss": 5.0040023624897}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.93668603897095, "eval_loss": 4.735761545598507, "generalization_gap": 0.6215345710515976, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.114226974546909, "train_loss_last": 4.047104835510254, "val_eval_loss": 4.735761545598507}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.82690095901489, "eval_loss": 4.546834863722324, "generalization_gap": 0.3490751311182976, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.197759732604027, "train_loss_last": 4.192391395568848, "val_eval_loss": 4.546834863722324}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.074, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.93229603767395, "eval_loss": 5.513543851673603, "generalization_gap": 1.3063586726784706, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.2071851789951324, "train_loss_last": 4.27359676361084, "val_eval_loss": 5.513543851673603}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.026999999999999996, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.69331002235413, "eval_loss": 5.339767083525658, "generalization_gap": 1.5578529387712479, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.78191414475441, "train_loss_last": 4.1406779289245605, "val_eval_loss": 5.339767083525658}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.69603514671326, "eval_loss": 5.008033439517021, "generalization_gap": 1.1422570124268532, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.865776427090168, "train_loss_last": 4.167153835296631, "val_eval_loss": 5.008033439517021}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.6384220123291, "eval_loss": 4.721928343176842, "generalization_gap": 0.67343620210886, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.048492141067982, "train_loss_last": 4.169475555419922, "val_eval_loss": 4.721928343176842}
+{"condition": "no_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.074, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.50973510742188, "eval_loss": 4.545698016881943, "generalization_gap": 0.3496076166629791, "model_config": {"block_size": 128, "dropout": 0.074, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.196090400218964, "train_loss_last": 4.262007713317871, "val_eval_loss": 4.545698016881943}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.187, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.34255576133728, "eval_loss": 5.484160475432873, "generalization_gap": 0.9717355743050575, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.512424901127815, "train_loss_last": 4.730109691619873, "val_eval_loss": 5.484160475432873}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.141, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.2273588180542, "eval_loss": 5.133235886693001, "generalization_gap": 1.0404804795980453, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.0927554070949554, "train_loss_last": 4.169230937957764, "val_eval_loss": 5.133235886693001}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.079, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.25440692901611, "eval_loss": 4.85265301913023, "generalization_gap": 0.7842404395341873, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.068412579596043, "train_loss_last": 4.261360168457031, "val_eval_loss": 4.85265301913023}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.020000000000000004, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.35925078392029, "eval_loss": 4.669060342013836, "generalization_gap": 0.5453823804855347, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.123677961528301, "train_loss_last": 4.134852886199951, "val_eval_loss": 4.669060342013836}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.29016304016113, "eval_loss": 4.507524698972702, "generalization_gap": 0.32788223028182983, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.179642468690872, "train_loss_last": 4.105343818664551, "val_eval_loss": 4.507524698972702}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.187, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.22968816757202, "eval_loss": 5.488361656665802, "generalization_gap": 0.9708076566457748, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.517554000020027, "train_loss_last": 4.531530857086182, "val_eval_loss": 5.488361656665802}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.141, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.1893401145935, "eval_loss": 5.109653554856777, "generalization_gap": 0.9958972632884979, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.113756291568279, "train_loss_last": 4.256237983703613, "val_eval_loss": 5.109653554856777}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.079, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.1724317073822, "eval_loss": 4.867041394114494, "generalization_gap": 0.7559577599167824, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.111083634197712, "train_loss_last": 4.158487319946289, "val_eval_loss": 4.867041394114494}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.020000000000000004, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.08005499839783, "eval_loss": 4.653611704707146, "generalization_gap": 0.5149502381682396, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.138661466538906, "train_loss_last": 4.011228561401367, "val_eval_loss": 4.653611704707146}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.10033988952637, "eval_loss": 4.5087380558252335, "generalization_gap": 0.3276814669370651, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.181056588888168, "train_loss_last": 4.2869648933410645, "val_eval_loss": 4.5087380558252335}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.187, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.04283380508423, "eval_loss": 5.460177332162857, "generalization_gap": 1.01016403734684, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.450013294816017, "train_loss_last": 4.721713066101074, "val_eval_loss": 5.460177332162857}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.141, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 84.09177088737488, "eval_loss": 5.123330071568489, "generalization_gap": 0.9677676260471344, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.155562445521355, "train_loss_last": 4.447000503540039, "val_eval_loss": 5.123330071568489}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.079, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 89.38473176956177, "eval_loss": 4.860383331775665, "generalization_gap": 0.794602245092392, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.065781086683273, "train_loss_last": 4.254539489746094, "val_eval_loss": 4.860383331775665}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.020000000000000004, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 95.26978206634521, "eval_loss": 4.671414420008659, "generalization_gap": 0.5674415826797485, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.103972837328911, "train_loss_last": 3.837702989578247, "val_eval_loss": 4.671414420008659}
+{"condition": "half_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.187, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 96.84964084625244, "eval_loss": 4.5001466274261475, "generalization_gap": 0.32612571120262146, "model_config": {"block_size": 128, "dropout": 0.187, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.174020916223526, "train_loss_last": 4.238502502441406, "val_eval_loss": 4.5001466274261475}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.415, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 97.09276008605957, "eval_loss": 5.625491619110107, "generalization_gap": 0.6792069971561432, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.946284621953964, "train_loss_last": 5.010383129119873, "val_eval_loss": 5.625491619110107}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.368, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.20200800895691, "eval_loss": 5.2015610486269, "generalization_gap": 0.6658378839492798, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.53572316467762, "train_loss_last": 4.839486122131348, "val_eval_loss": 5.2015610486269}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.275, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.62430381774902, "eval_loss": 4.8964985981583595, "generalization_gap": 0.5092504248023033, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.387248173356056, "train_loss_last": 4.6044464111328125, "val_eval_loss": 4.8964985981583595}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.163, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.15935587882996, "eval_loss": 4.68208322674036, "generalization_gap": 0.3359387591481209, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.346144467592239, "train_loss_last": 4.363907337188721, "val_eval_loss": 4.68208322674036}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.041, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.86329984664917, "eval_loss": 4.497698821127415, "generalization_gap": 0.2438541278243065, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.253844693303108, "train_loss_last": 4.3459343910217285, "val_eval_loss": 4.497698821127415}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.415, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.03171682357788, "eval_loss": 5.6271611005067825, "generalization_gap": 0.7019565254449844, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.925204575061798, "train_loss_last": 5.111268997192383, "val_eval_loss": 5.6271611005067825}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.368, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.64983797073364, "eval_loss": 5.153126522898674, "generalization_gap": 0.636242464184761, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.516884058713913, "train_loss_last": 4.594588756561279, "val_eval_loss": 5.153126522898674}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.275, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.76931095123291, "eval_loss": 4.903263486921787, "generalization_gap": 0.481093131005764, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.422170355916023, "train_loss_last": 4.459939956665039, "val_eval_loss": 4.903263486921787}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.163, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.61855697631836, "eval_loss": 4.6622872948646545, "generalization_gap": 0.3234003335237503, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.338886961340904, "train_loss_last": 4.262211322784424, "val_eval_loss": 4.6622872948646545}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.041, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.36984968185425, "eval_loss": 4.4968860894441605, "generalization_gap": 0.23646819591522217, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 2, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.260417893528938, "train_loss_last": 4.202493667602539, "val_eval_loss": 4.4968860894441605}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.415, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.81767106056213, "eval_loss": 5.601569518446922, "generalization_gap": 0.7185685783624649, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.883000940084457, "train_loss_last": 5.0489606857299805, "val_eval_loss": 5.601569518446922}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.368, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 90.1187698841095, "eval_loss": 5.196595400571823, "generalization_gap": 0.625303328037262, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.571292072534561, "train_loss_last": 4.7599873542785645, "val_eval_loss": 5.196595400571823}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.275, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 86.78390669822693, "eval_loss": 4.898942299187183, "generalization_gap": 0.5071352049708366, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.391807094216347, "train_loss_last": 4.561906814575195, "val_eval_loss": 4.898942299187183}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.163, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 86.44374418258667, "eval_loss": 4.672604776918888, "generalization_gap": 0.3575567975640297, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.315047979354858, "train_loss_last": 4.3315110206604, "val_eval_loss": 4.672604776918888}
+{"condition": "high_sample_pressure_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.041, "dropout_final": 0.041, "dropout_initial": 0.415, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 86.21241497993469, "eval_loss": 4.492968052625656, "generalization_gap": 0.24503198266029358, "model_config": {"block_size": 128, "dropout": 0.415, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 3, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.2479360699653625, "train_loss_last": 4.288923263549805, "val_eval_loss": 4.492968052625656}

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/summary.csv

@@ -0,0 +1,16 @@
+run_mode,condition,condition_kind,stage,token_limit,model_name,n_layer,n_head,n_embd,parameters,dropout_initial,dropout_final,dropout_schedule,n,mean_train_eval_loss,std_train_eval_loss,mean_val_eval_loss,std_val_eval_loss,mean_generalization_gap,std_generalization_gap
+locked_stream,half_sample_pressure_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.187,0.02,log_prefix_anchor,3,4.493330731987953,0.0376015582951025,5.477566488087177,0.015205247079814913,0.9842357560992241,0.022459342914190557
+locked_stream,high_sample_pressure_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.415,0.041,log_prefix_anchor,3,4.918163379033406,0.03222405823096952,5.618074079354604,0.0143177229369755,0.6999107003211975,0.019760378768973613
+locked_stream,no_sample_pressure_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.074,0.02,log_prefix_anchor,3,4.226404994726181,0.022079363458691863,5.5298768157760305,0.014693288208937989,1.3034718210498493,0.01865235131123608
+locked_stream,half_sample_pressure_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.187,0.02,log_prefix_anchor,3,4.120691381394863,0.031972685563632086,5.122073171039422,0.011841302416263908,1.0013817896445592,0.03666537590004777
+locked_stream,high_sample_pressure_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.415,0.041,log_prefix_anchor,3,4.541299765308698,0.027629365680845203,5.183760990699132,0.026646151283341232,0.6424612253904343,0.02097062943609501
+locked_stream,no_sample_pressure_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.074,0.02,log_prefix_anchor,3,3.7632730826735497,0.01823977881782879,5.3265082860986395,0.012804760577866508,1.5632352034250896,0.0054491573301392775
+locked_stream,half_sample_pressure_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.187,0.02,log_prefix_anchor,3,4.081759100159009,0.025429852755723174,4.860025915006797,0.007200843265337238,0.7782668148477873,0.02000280486687939
+locked_stream,high_sample_pressure_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.415,0.041,log_prefix_anchor,3,4.400408541162808,0.0189836344219835,4.89956812808911,0.0034255913878121495,0.4991595869263013,0.015681714220084353
+locked_stream,no_sample_pressure_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.074,0.02,log_prefix_anchor,3,3.9007718513409295,0.03475500986994197,5.0044328446189565,0.003405819537978033,1.1036609932780266,0.03690407595201124
+locked_stream,half_sample_pressure_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.187,0.02,log_prefix_anchor,3,4.122104088465373,0.017397788837400955,4.664695488909881,0.00967073549379882,0.5425914004445076,0.026356735179997894
+locked_stream,high_sample_pressure_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.415,0.041,log_prefix_anchor,3,4.333359802762668,0.01626837443127391,4.672325099507968,0.009900928961341445,0.3389652967453003,0.01727819308562328
+locked_stream,no_sample_pressure_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.074,0.02,log_prefix_anchor,3,4.084012111028035,0.033186973331260146,4.733480711778005,0.01059765965680393,0.6494686007499695,0.026177170242985152
+locked_stream,half_sample_pressure_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.187,0.02,log_prefix_anchor,3,4.178239991267522,0.003721609820125387,4.505469794074695,0.004649745809078966,0.3272298028071721,0.0009614261196679384
+locked_stream,high_sample_pressure_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.415,0.041,log_prefix_anchor,3,4.254066218932469,0.006243859789537772,4.49585098773241,0.002529549232483656,0.24178476879994074,0.004641798781998004
+locked_stream,no_sample_pressure_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.074,0.02,log_prefix_anchor,3,4.198599298795064,0.003017586763546228,4.546837501227856,0.0011408053853665143,0.3482382024327914,0.0019292089078293878

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/summary.json

@@ -0,0 +1,332 @@
+[
+  {
+    "run_mode": "locked_stream",
+    "condition": "half_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.187,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.493330731987953,
+    "std_train_eval_loss": 0.0376015582951025,
+    "mean_val_eval_loss": 5.477566488087177,
+    "std_val_eval_loss": 0.015205247079814913,
+    "mean_generalization_gap": 0.9842357560992241,
+    "std_generalization_gap": 0.022459342914190557
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "high_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.415,
+    "dropout_final": 0.041,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.918163379033406,
+    "std_train_eval_loss": 0.03222405823096952,
+    "mean_val_eval_loss": 5.618074079354604,
+    "std_val_eval_loss": 0.0143177229369755,
+    "mean_generalization_gap": 0.6999107003211975,
+    "std_generalization_gap": 0.019760378768973613
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "no_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.074,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.226404994726181,
+    "std_train_eval_loss": 0.022079363458691863,
+    "mean_val_eval_loss": 5.5298768157760305,
+    "std_val_eval_loss": 0.014693288208937989,
+    "mean_generalization_gap": 1.3034718210498493,
+    "std_generalization_gap": 0.01865235131123608
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "half_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.187,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.120691381394863,
+    "std_train_eval_loss": 0.031972685563632086,
+    "mean_val_eval_loss": 5.122073171039422,
+    "std_val_eval_loss": 0.011841302416263908,
+    "mean_generalization_gap": 1.0013817896445592,
+    "std_generalization_gap": 0.03666537590004777
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "high_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.415,
+    "dropout_final": 0.041,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.541299765308698,
+    "std_train_eval_loss": 0.027629365680845203,
+    "mean_val_eval_loss": 5.183760990699132,
+    "std_val_eval_loss": 0.026646151283341232,
+    "mean_generalization_gap": 0.6424612253904343,
+    "std_generalization_gap": 0.02097062943609501
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "no_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.074,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 3.7632730826735497,
+    "std_train_eval_loss": 0.01823977881782879,
+    "mean_val_eval_loss": 5.3265082860986395,
+    "std_val_eval_loss": 0.012804760577866508,
+    "mean_generalization_gap": 1.5632352034250896,
+    "std_generalization_gap": 0.0054491573301392775
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "half_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.187,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.081759100159009,
+    "std_train_eval_loss": 0.025429852755723174,
+    "mean_val_eval_loss": 4.860025915006797,
+    "std_val_eval_loss": 0.007200843265337238,
+    "mean_generalization_gap": 0.7782668148477873,
+    "std_generalization_gap": 0.02000280486687939
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "high_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.415,
+    "dropout_final": 0.041,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.400408541162808,
+    "std_train_eval_loss": 0.0189836344219835,
+    "mean_val_eval_loss": 4.89956812808911,
+    "std_val_eval_loss": 0.0034255913878121495,
+    "mean_generalization_gap": 0.4991595869263013,
+    "std_generalization_gap": 0.015681714220084353
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "no_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.074,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 3.9007718513409295,
+    "std_train_eval_loss": 0.03475500986994197,
+    "mean_val_eval_loss": 5.0044328446189565,
+    "std_val_eval_loss": 0.003405819537978033,
+    "mean_generalization_gap": 1.1036609932780266,
+    "std_generalization_gap": 0.03690407595201124
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "half_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.187,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.122104088465373,
+    "std_train_eval_loss": 0.017397788837400955,
+    "mean_val_eval_loss": 4.664695488909881,
+    "std_val_eval_loss": 0.00967073549379882,
+    "mean_generalization_gap": 0.5425914004445076,
+    "std_generalization_gap": 0.026356735179997894
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "high_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.415,
+    "dropout_final": 0.041,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.333359802762668,
+    "std_train_eval_loss": 0.01626837443127391,
+    "mean_val_eval_loss": 4.672325099507968,
+    "std_val_eval_loss": 0.009900928961341445,
+    "mean_generalization_gap": 0.3389652967453003,
+    "std_generalization_gap": 0.01727819308562328
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "no_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.074,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.084012111028035,
+    "std_train_eval_loss": 0.033186973331260146,
+    "mean_val_eval_loss": 4.733480711778005,
+    "std_val_eval_loss": 0.01059765965680393,
+    "mean_generalization_gap": 0.6494686007499695,
+    "std_generalization_gap": 0.026177170242985152
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "half_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.187,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.178239991267522,
+    "std_train_eval_loss": 0.003721609820125387,
+    "mean_val_eval_loss": 4.505469794074695,
+    "std_val_eval_loss": 0.004649745809078966,
+    "mean_generalization_gap": 0.3272298028071721,
+    "std_generalization_gap": 0.0009614261196679384
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "high_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.415,
+    "dropout_final": 0.041,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.254066218932469,
+    "std_train_eval_loss": 0.006243859789537772,
+    "mean_val_eval_loss": 4.49585098773241,
+    "std_val_eval_loss": 0.002529549232483656,
+    "mean_generalization_gap": 0.24178476879994074,
+    "std_generalization_gap": 0.004641798781998004
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "no_sample_pressure_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.074,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 3,
+    "mean_train_eval_loss": 4.198599298795064,
+    "std_train_eval_loss": 0.003017586763546228,
+    "mean_val_eval_loss": 4.546837501227856,
+    "std_val_eval_loss": 0.0011408053853665143,
+    "mean_generalization_gap": 0.3482382024327914,
+    "std_generalization_gap": 0.0019292089078293878
+  }
+]

runs/formula_l12_sample_pressure_ablation/locked_stream/20260528-053842/trace.jsonl

@@ -0,0 +1,90 @@
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.288013458251953}
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.3242878913879395}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.188028335571289}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.084026336669922}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.080935478210449}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.7913968563079834}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.364566802978516}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.254018306732178}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.194523811340332}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.2256083488464355}
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.205361366271973}
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.35040283203125}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.05342960357666}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.9610917568206787}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.236403942108154}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.683029890060425}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.517946243286133}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.047104835510254}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.297525882720947}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.192391395568848}
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.332969665527344}
+{"condition": "no_sample_pressure_l12", "dropout": 0.074, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.27359676361084}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.443536758422852}
+{"condition": "no_sample_pressure_l12", "dropout": 0.026999999999999996, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.1406779289245605}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.314910888671875}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.167153835296631}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.215195655822754}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.169475555419922}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.255288124084473}
+{"condition": "no_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.262007713317871}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.11745548248291}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.730109691619873}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.677674770355225}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.169230937957764}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.304385185241699}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.261360168457031}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.337337017059326}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.134852886199951}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.236599445343018}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.105343818664551}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.420967102050781}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.531530857086182}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.597446441650391}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.256237983703613}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.313892364501953}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.158487319946289}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.25571346282959}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.011228561401367}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.045015335083008}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.2869648933410645}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.417287826538086}
+{"condition": "half_sample_pressure_l12", "dropout": 0.187, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.721713066101074}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.639064788818359}
+{"condition": "half_sample_pressure_l12", "dropout": 0.141, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.447000503540039}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.416566848754883}
+{"condition": "half_sample_pressure_l12", "dropout": 0.079, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.254539489746094}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.4399566650390625}
+{"condition": "half_sample_pressure_l12", "dropout": 0.020000000000000004, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.837702989578247}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.111726760864258}
+{"condition": "half_sample_pressure_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.238502502441406}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.778111457824707}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 5.010383129119873}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.981586456298828}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.839486122131348}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.571419715881348}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.6044464111328125}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.515482425689697}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.363907337188721}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.33910608291626}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.3459343910217285}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.666861534118652}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 5.111268997192383}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.890622138977051}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.594588756561279}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.701104640960693}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.459939956665039}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.701330184936523}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.262211322784424}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.375853538513184}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 2, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.202493667602539}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.75937557220459}
+{"condition": "high_sample_pressure_l12", "dropout": 0.415, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 5.0489606857299805}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.861799240112305}
+{"condition": "high_sample_pressure_l12", "dropout": 0.368, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.7599873542785645}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.9176530838012695}
+{"condition": "high_sample_pressure_l12", "dropout": 0.275, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.561906814575195}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.551490783691406}
+{"condition": "high_sample_pressure_l12", "dropout": 0.163, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.3315110206604}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.285399436950684}
+{"condition": "high_sample_pressure_l12", "dropout": 0.041, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 3, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.288923263549805}

src/dropout_decay/experiment.py

@@ -1649,7 +1649,7 @@ def build_parser() -> argparse.ArgumentParser:
     parser.add_argument("--batch-size", type=int, default=16)
     parser.add_argument("--steps", type=int, default=2000)
     parser.add_argument("--stage-steps", type=int, default=1000)
-    parser.add_argument("--dropout-rates", nargs="+", type=float, default=DEFAULT_DROPOUT_RATES)
+    parser.add_argument("--dropout-rates", nargs="*", type=float, default=DEFAULT_DROPOUT_RATES)
     parser.add_argument("--decays", nargs="*", type=parse_decay_spec, default=[])
     parser.add_argument(
         "--anchor-decays",