Add probe-calibrated streaming results

README.md

@@ -72,7 +72,9 @@ explanation of the hypothesis, math, evidence, and current limits, see
 `docs/dropout_decay_hypothesis_summary.md`. For the denser run-by-run research
 summary, see `docs/dropout_decay_research_report_v2.md`. For the corpus
 difficulty follow-up that motivated the next formula refinement, see
-`docs/corpus_difficulty_probe_20260529.md`.
+`docs/corpus_difficulty_probe_20260529.md`. For the first single-seed
+probe-calibrated streaming test, see
+`docs/probe_calibrated_stream_20260529.md`.
 
 ## Step 1: Cheap Static Screen
 

docs/dropout_decay_hypothesis_summary.md

@@ -720,3 +720,22 @@ This strengthens the need for a corpus term, but also shows that raw
 train-validation gap is not enough: WikiText-103 had the largest gap while still
 preferring near-zero dropout in this probe. The most defensible next refinement
 is a cheap calibration-probe term rather than a pure closed-form corpus statistic.
+
+## 16. Post-Report Addendum: Probe-Calibrated Streaming
+
+A first single-seed streaming test of probe-calibrated schedules is saved in
+`docs/probe_calibrated_stream_20260529.md`.
+
+Key result:
+
+| Corpus | Best final schedule in this test | Final val | Mean trajectory |
+|---|---|---:|---:|
+| Local | Old formula | 4.4834 | 4.9249 |
+| WikiText-103 | Old formula | 4.0643 | 4.5704 |
+| TinyStories | Low-decay heuristic | 2.7068 | 3.0793 |
+
+The strict probe-calibrated formula was too low on every corpus. The blended
+probe-calibrated formula improved the mean trajectory on local and WikiText-103
+and improved both final and trajectory on TinyStories versus the old formula.
+This supports using the probe as a damped downward correction to the old pressure
+formula, not as a direct replacement for target-model dropout scaling.

docs/probe_calibrated_stream_20260529.md

@@ -0,0 +1,137 @@
+# Probe-Calibrated Streaming Test - 2026-05-29
+
+This note records the first single-seed streaming test of probe-calibrated
+dropout schedules. It follows the corpus-difficulty probe in
+`docs/corpus_difficulty_probe_20260529.md`.
+
+All runs used MPS only.
+
+## Target Setup
+
+| Field | Value |
+|---|---:|
+| Model | `L12_H8_D320` |
+| Parameters | 17,367,040 |
+| Seeds | 1 |
+| Prefixes | 250k, 500k, 1M, 2M, 4M |
+| Stage steps | 1000 |
+| Batch size | 16 |
+| Block size | 128 |
+| Tokens per stage | 2.048M |
+
+## Tested Schedules
+
+The old pressure formula schedule was:
+
+```text
+0.301 -> 0.255 -> 0.177 -> 0.087 -> 0.020
+```
+
+The strict probe-calibrated schedule used the measured `p_probe` as a direct
+corpus-specific intercept correction. This was intentionally aggressive.
+
+The blended probe-calibrated schedule averaged the old formula anchors with the
+strict anchors. This tests whether the probe is useful as a correction signal
+rather than as a complete replacement for target-model scaling.
+
+| Corpus | Strict schedule | Blended schedule |
+|---|---|---|
+| Local | `0.115 -> 0.069 -> 0.000 -> 0.000 -> 0.000` | `0.208 -> 0.162 -> 0.089 -> 0.044 -> 0.010` |
+| WikiText-103 | `0.075 -> 0.029 -> 0.000 -> 0.000 -> 0.000` | `0.188 -> 0.142 -> 0.089 -> 0.044 -> 0.010` |
+| TinyStories | `0.095 -> 0.049 -> 0.000 -> 0.000 -> 0.000` | `0.198 -> 0.152 -> 0.089 -> 0.044 -> 0.010` |
+
+The hand low-decay heuristic was:
+
+```text
+0.140 -> 0.140 -> 0.100 -> 0.060 -> 0.020
+```
+
+## Run Artifacts
+
+| Corpus | Run path |
+|---|---|
+| Local | `runs/probe_calibrated_stream_local/locked_stream/20260529-093748` |
+| WikiText-103 | `runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250` |
+| TinyStories | `runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913` |
+
+## Summary Results
+
+| Corpus | Condition | Final val | Mean trajectory | Active dropouts |
+|---|---|---:|---:|---|
+| Local | Old formula | 4.4834 | 4.9249 | `0.301 -> 0.255 -> 0.177 -> 0.087 -> 0.020` |
+| Local | Probe blend | 4.5024 | 4.9214 | `0.208 -> 0.162 -> 0.089 -> 0.044 -> 0.010` |
+| Local | Probe strict | 4.5605 | 5.0136 | `0.115 -> 0.069 -> 0.000 -> 0.000 -> 0.000` |
+| Local | Low-decay heuristic | 4.5063 | 4.9360 | `0.140 -> 0.140 -> 0.100 -> 0.060 -> 0.020` |
+| Local | Best static | 4.5205 | 4.9364 | `0.140` |
+| WikiText-103 | Old formula | 4.0643 | 4.5704 | `0.301 -> 0.255 -> 0.177 -> 0.087 -> 0.020` |
+| WikiText-103 | Probe blend | 4.0689 | 4.5557 | `0.188 -> 0.142 -> 0.089 -> 0.044 -> 0.010` |
+| WikiText-103 | Probe strict | 4.1441 | 4.6759 | `0.075 -> 0.029 -> 0.000 -> 0.000 -> 0.000` |
+| WikiText-103 | Low-decay heuristic | 4.0854 | 4.5683 | `0.140 -> 0.140 -> 0.100 -> 0.060 -> 0.020` |
+| WikiText-103 | Best static | 4.0893 | 4.5889 | `0.080` |
+| TinyStories | Old formula | 2.7221 | 3.1149 | `0.301 -> 0.255 -> 0.177 -> 0.087 -> 0.020` |
+| TinyStories | Probe blend | 2.7194 | 3.0934 | `0.198 -> 0.152 -> 0.089 -> 0.044 -> 0.010` |
+| TinyStories | Probe strict | 2.7312 | 3.1178 | `0.095 -> 0.049 -> 0.000 -> 0.000 -> 0.000` |
+| TinyStories | Low-decay heuristic | 2.7068 | 3.0793 | `0.140 -> 0.140 -> 0.100 -> 0.060 -> 0.020` |
+| TinyStories | Best static | 2.7215 | 3.0880 | `0.080` |
+
+## Interpretation
+
+The strict schedule is too low. It improves the first TinyStories stages but
+collapses to zero dropout too early and loses final loss on all three corpora.
+This rules out the simplest use of `p_probe` as a direct target-model dropout
+anchor.
+
+The blended schedule is useful:
+
+- Local: best mean trajectory, but old formula wins final.
+- WikiText-103: best mean trajectory, but old formula wins final by a tiny
+  margin.
+- TinyStories: beats old formula on both final loss and mean trajectory.
+
+The hand low-decay heuristic remains the best TinyStories schedule in this
+single seed. This means the calibration direction is right, but the current
+blend underestimates the useful midstream dropout for TinyStories. The next
+formula should pull down the old formula for easier corpora, but not as sharply
+as the strict probe-corrected formula.
+
+## Formula Consequence
+
+The evidence supports this framing:
+
+```text
+dropout_schedule = old_pressure_schedule + corpus_correction
+```
+
+Not this:
+
+```text
+dropout_schedule = direct_probe_optimum_scaled_to_target
+```
+
+A reasonable next candidate is a damped correction:
+
+```text
+p_t = clamp(p_min, p_max,
+            p_old_t
+          - lambda * max(0, p_old_probe_equivalent - p_probe)
+          * g(U_t, C_t))
+```
+
+Where:
+
+- `p_old_t` is the old pressure-formula dropout at stream stage `t`.
+- `p_probe` is the measured warmup optimum.
+- `p_old_probe_equivalent` is what the old formula would have predicted for the
+  standardized probe setting.
+- `lambda` should be less than 1.
+- `g(U_t, C_t)` should decay the correction with stream progress, so the probe
+  mostly adjusts early and midstream dropout.
+
+The practical recommendation after this test:
+
+1. Keep the old pressure formula as the base schedule.
+2. Use the calibration probe as a downward correction signal.
+3. Fit a damped corpus correction coefficient against local, WikiText-103, and
+   TinyStories.
+4. Validate the fitted correction with 3 seeds before making a paper claim.
+

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/RESULT_SUMMARY.md

@@ -0,0 +1,98 @@
+# Locked Streaming Dropout Summary
+
+Run directory: `runs/probe_calibrated_stream_local/locked_stream/20260529-093748`
+
+Model: `L12_H8_D320` causal Transformer, 17,367,040 parameters, 12 layers, 8 heads, 320 embedding dim.
+Training per stage: 1,000 steps. Sampled tokens are cumulative in each stage row. Seeds present: 1.
+
+## Condition Ranking
+
+| Condition | Kind | Final dropout | Mean trajectory val loss | Final val loss | Final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---|
+| `probe_cal_blend_local` | anchor_decay | 0.01 | 4.9214 | 4.5024 | 0.3367 | 0.21 -> 0.16 -> 0.09 -> 0.04 -> 0.01 |
+| `old_formula_l12` | anchor_decay | 0.02 | 4.9249 | 4.4834 | 0.2656 | 0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02 |
+| `static_dropout_0.2` | static | 0.20 | 4.9341 | 4.5339 | 0.2261 | 0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20 |
+| `low_decay_heuristic` | anchor_decay | 0.02 | 4.9360 | 4.5063 | 0.3051 | 0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02 |
+| `static_dropout_0.14` | static | 0.14 | 4.9364 | 4.5205 | 0.2366 | 0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14 |
+| `static_dropout_0.08` | static | 0.08 | 4.9776 | 4.5278 | 0.2738 | 0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08 |
+| `probe_cal_strict_local` | anchor_decay | 0.00 | 5.0136 | 4.5605 | 0.3787 | 0.12 -> 0.07 -> 0.00 -> 0.00 -> 0.00 |
+| `static_dropout_0.04` | static | 0.04 | 5.0339 | 4.5395 | 0.3324 | 0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04 |
+| `static_dropout_0.02` | static | 0.02 | 5.0838 | 4.5625 | 0.3491 | 0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02 |
+| `static_dropout_0` | static | 0.00 | 5.1667 | 4.6072 | 0.4137 | 0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00 |
+
+## Stage Trajectory
+
+### Stage 0: 250,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `static_dropout_0.2` | 0.20 | 5.4850 | 4.5417 | 0.9433 | 1 |
+| `probe_cal_blend_local` | 0.21 | 5.4906 | 4.5522 | 0.9385 | 1 |
+| `low_decay_heuristic` | 0.14 | 5.4954 | 4.4045 | 1.0909 | 1 |
+| `static_dropout_0.14` | 0.14 | 5.4954 | 4.4045 | 1.0909 | 1 |
+| `probe_cal_strict_local` | 0.12 | 5.5104 | 4.3629 | 1.1474 | 1 |
+| `old_formula_l12` | 0.30 | 5.5306 | 4.7508 | 0.7797 | 1 |
+| `static_dropout_0.08` | 0.08 | 5.5353 | 4.2603 | 1.2750 | 1 |
+| `static_dropout_0.04` | 0.04 | 5.6080 | 4.1403 | 1.4676 | 1 |
+| `static_dropout_0.02` | 0.02 | 5.6287 | 4.0796 | 1.5492 | 1 |
+| `static_dropout_0` | 0.00 | 5.7414 | 3.9755 | 1.7659 | 1 |
+
+### Stage 1: 500,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `probe_cal_blend_local` | 0.16 | 5.1257 | 4.1697 | 0.9560 | 1 |
+| `static_dropout_0.2` | 0.20 | 5.1344 | 4.2157 | 0.9187 | 1 |
+| `old_formula_l12` | 0.26 | 5.1356 | 4.3445 | 0.7912 | 1 |
+| `low_decay_heuristic` | 0.14 | 5.1448 | 4.0911 | 1.0537 | 1 |
+| `static_dropout_0.14` | 0.14 | 5.1448 | 4.0911 | 1.0537 | 1 |
+| `probe_cal_strict_local` | 0.07 | 5.2213 | 3.9278 | 1.2935 | 1 |
+| `static_dropout_0.08` | 0.08 | 5.2301 | 3.9256 | 1.3044 | 1 |
+| `static_dropout_0.04` | 0.04 | 5.3272 | 3.7610 | 1.5662 | 1 |
+| `static_dropout_0.02` | 0.02 | 5.4186 | 3.7063 | 1.7124 | 1 |
+| `static_dropout_0` | 0.00 | 5.5227 | 3.5902 | 1.9324 | 1 |
+
+### Stage 2: 1,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `old_formula_l12` | 0.18 | 4.8342 | 4.2493 | 0.5848 | 1 |
+| `probe_cal_blend_local` | 0.09 | 4.8391 | 4.0901 | 0.7490 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.8441 | 4.2444 | 0.5997 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.8538 | 4.1499 | 0.7039 | 1 |
+| `low_decay_heuristic` | 0.10 | 4.8597 | 4.0948 | 0.7649 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.9009 | 4.0427 | 0.8582 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.9678 | 3.9501 | 1.0177 | 1 |
+| `probe_cal_strict_local` | 0.00 | 5.0119 | 3.9013 | 1.1106 | 1 |
+| `static_dropout_0.02` | 0.02 | 5.0315 | 3.8937 | 1.1378 | 1 |
+| `static_dropout_0` | 0.00 | 5.1371 | 3.8068 | 1.3302 | 1 |
+
+### Stage 3: 2,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `old_formula_l12` | 0.09 | 4.6408 | 4.2222 | 0.4186 | 1 |
+| `probe_cal_blend_local` | 0.04 | 4.6491 | 4.1580 | 0.4912 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.6676 | 4.2630 | 0.4046 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.6731 | 4.3232 | 0.3499 | 1 |
+| `low_decay_heuristic` | 0.06 | 4.6737 | 4.1929 | 0.4809 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.6937 | 4.1818 | 0.5119 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.7271 | 4.1270 | 0.6001 | 1 |
+| `probe_cal_strict_local` | 0.00 | 4.7637 | 4.0890 | 0.6747 | 1 |
+| `static_dropout_0.02` | 0.02 | 4.7777 | 4.1062 | 0.6715 | 1 |
+| `static_dropout_0` | 0.00 | 4.8252 | 4.0797 | 0.7456 | 1 |
+
+### Stage 4: 4,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `old_formula_l12` | 0.02 | 4.4834 | 4.2178 | 0.2656 | 1 |
+| `probe_cal_blend_local` | 0.01 | 4.5024 | 4.1657 | 0.3367 | 1 |
+| `low_decay_heuristic` | 0.02 | 4.5063 | 4.2012 | 0.3051 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.5205 | 4.2840 | 0.2366 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.5278 | 4.2540 | 0.2738 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.5339 | 4.3079 | 0.2261 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.5395 | 4.2072 | 0.3324 | 1 |
+| `probe_cal_strict_local` | 0.00 | 4.5605 | 4.1818 | 0.3787 | 1 |
+| `static_dropout_0.02` | 0.02 | 4.5625 | 4.2134 | 0.3491 | 1 |
+| `static_dropout_0` | 0.00 | 4.6072 | 4.1935 | 0.4137 | 1 |

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/config.json

@@ -0,0 +1,216 @@
+{
+  "args": {
+    "mode": "locked_stream",
+    "corpus": null,
+    "corpus_glob": null,
+    "text_column": "text",
+    "use_cached_data": true,
+    "output_dir": "runs/probe_calibrated_stream_local",
+    "resume_from": null,
+    "cache_dir": ".cache/dropout_decay",
+    "models": [
+      "L12_H8_D320=12x8x320"
+    ],
+    "seeds": [
+      1
+    ],
+    "token_limits": [
+      5000000
+    ],
+    "stream_token_caps": [
+      250000,
+      500000,
+      1000000,
+      2000000,
+      4000000
+    ],
+    "val_tokens": 500000,
+    "allow_short_corpus": false,
+    "force_retokenize": false,
+    "vocab_size": 4096,
+    "tokenizer_train_chars": 10000000,
+    "block_size": 128,
+    "batch_size": 16,
+    "steps": 2000,
+    "stage_steps": 1000,
+    "dropout_rates": [
+      0.0,
+      0.02,
+      0.04,
+      0.08,
+      0.14,
+      0.2
+    ],
+    "decays": [],
+    "anchor_decays": [
+      {
+        "name": "old_formula_l12",
+        "kind": "anchor_decay",
+        "initial": 0.301,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.301
+          ],
+          [
+            500000,
+            0.255
+          ],
+          [
+            1000000,
+            0.177
+          ],
+          [
+            2000000,
+            0.087
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_strict_local",
+        "kind": "anchor_decay",
+        "initial": 0.115,
+        "final": 0.0,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.115
+          ],
+          [
+            500000,
+            0.069
+          ],
+          [
+            1000000,
+            0.0
+          ],
+          [
+            2000000,
+            0.0
+          ],
+          [
+            4000000,
+            0.0
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_blend_local",
+        "kind": "anchor_decay",
+        "initial": 0.208,
+        "final": 0.01,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.208
+          ],
+          [
+            500000,
+            0.162
+          ],
+          [
+            1000000,
+            0.089
+          ],
+          [
+            2000000,
+            0.044
+          ],
+          [
+            4000000,
+            0.01
+          ]
+        ]
+      },
+      {
+        "name": "low_decay_heuristic",
+        "kind": "anchor_decay",
+        "initial": 0.14,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.14
+          ],
+          [
+            500000,
+            0.14
+          ],
+          [
+            1000000,
+            0.1
+          ],
+          [
+            2000000,
+            0.06
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      }
+    ],
+    "decay_tokens": null,
+    "eval_batches": 64,
+    "train_eval_batches": 32,
+    "trace_eval_batches": 8,
+    "eval_every": 0,
+    "log_every": 500,
+    "lr": 0.0003,
+    "weight_decay": 0.1,
+    "grad_clip": 1.0,
+    "plateau_delta": 0.01,
+    "target_min_dropout": 0.1,
+    "min_nonzero_margin": 0.01,
+    "min_high_dropout_margin": 0.03,
+    "screen_early_stop": false,
+    "screen_prune_patience": 3,
+    "screen_prune_min_delta": 0.01
+  },
+  "mode": "locked_stream",
+  "seeds": [
+    1
+  ],
+  "models": [
+    {
+      "model_name": "L12_H8_D320",
+      "n_layer": 12,
+      "n_head": 8,
+      "n_embd": 320
+    }
+  ],
+  "device": "mps",
+  "torch": "2.12.0",
+  "python": "3.11.15 (main, Mar  3 2026, 00:52:57) [Clang 21.0.0 (clang-2100.0.123.102)]",
+  "mps_available": true,
+  "attribution": "Derived from Andrej Karpathy's nanochat project (https://github.com/karpathy/nanochat), MIT License, Copyright (c) 2025 Andrej Karpathy.",
+  "tokenizer_path": ".cache/dropout_decay/tokenizer-v4096.json",
+  "encoded_path": ".cache/dropout_decay/tokens-v4096-uint16.npy",
+  "train_tokens": 5000970,
+  "val_tokens": 500000,
+  "effective_token_limits": [
+    5000000
+  ],
+  "effective_stream_token_caps": [
+    250000,
+    500000,
+    1000000,
+    2000000,
+    4000000
+  ],
+  "resume_from": null
+}

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/metrics.jsonl

@@ -0,0 +1,50 @@
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.301, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 88.79629898071289, "eval_loss": 5.53056164085865, "generalization_gap": 0.77974072098732, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.75082091987133, "train_loss_last": 4.885682582855225, "val_eval_loss": 5.53056164085865}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.255, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 88.87150835990906, "eval_loss": 5.1356144696474075, "generalization_gap": 0.7911568731069565, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.344457596540451, "train_loss_last": 4.61283540725708, "val_eval_loss": 5.1356144696474075}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.177, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 88.80370712280273, "eval_loss": 4.834159925580025, "generalization_gap": 0.5848206281661987, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.249339297413826, "train_loss_last": 4.361461639404297, "val_eval_loss": 4.834159925580025}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.087, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 91.87760782241821, "eval_loss": 4.640849784016609, "generalization_gap": 0.418622650206089, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.22222713381052, "train_loss_last": 4.314380645751953, "val_eval_loss": 4.640849784016609}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 97.83016085624695, "eval_loss": 4.483362518250942, "generalization_gap": 0.26557936519384384, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.217783153057098, "train_loss_last": 4.028076171875, "val_eval_loss": 4.483362518250942}
+{"condition": "probe_cal_strict_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.115, "dropout_final": 0.0, "dropout_initial": 0.115, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.6944670677185, "eval_loss": 5.510357916355133, "generalization_gap": 1.1474400013685226, "model_config": {"block_size": 128, "dropout": 0.115, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.36291791498661, "train_loss_last": 4.475851535797119, "val_eval_loss": 5.510357916355133}
+{"condition": "probe_cal_strict_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.069, "dropout_final": 0.0, "dropout_initial": 0.115, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.90206098556519, "eval_loss": 5.221328005194664, "generalization_gap": 1.2935236766934395, "model_config": {"block_size": 128, "dropout": 0.115, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.9278043285012245, "train_loss_last": 4.225583076477051, "val_eval_loss": 5.221328005194664}
+{"condition": "probe_cal_strict_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.115, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 97.14144587516785, "eval_loss": 5.0118745639920235, "generalization_gap": 1.1105796843767166, "model_config": {"block_size": 128, "dropout": 0.115, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.901294879615307, "train_loss_last": 4.032376289367676, "val_eval_loss": 5.0118745639920235}
+{"condition": "probe_cal_strict_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.115, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 97.3501410484314, "eval_loss": 4.763735756278038, "generalization_gap": 0.6747253239154816, "model_config": {"block_size": 128, "dropout": 0.115, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.0890104323625565, "train_loss_last": 4.111223220825195, "val_eval_loss": 4.763735756278038}
+{"condition": "probe_cal_strict_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.115, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 97.69446778297424, "eval_loss": 4.560461081564426, "generalization_gap": 0.3786947801709175, "model_config": {"block_size": 128, "dropout": 0.115, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.181766301393509, "train_loss_last": 4.159558296203613, "val_eval_loss": 4.560461081564426}
+{"condition": "probe_cal_blend_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.208, "dropout_final": 0.01, "dropout_initial": 0.208, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 101.48727774620056, "eval_loss": 5.490625523030758, "generalization_gap": 0.9384564533829689, "model_config": {"block_size": 128, "dropout": 0.208, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.552169069647789, "train_loss_last": 4.479777812957764, "val_eval_loss": 5.490625523030758}
+{"condition": "probe_cal_blend_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.162, "dropout_final": 0.01, "dropout_initial": 0.208, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.11098217964172, "eval_loss": 5.125715062022209, "generalization_gap": 0.9560352489352226, "model_config": {"block_size": 128, "dropout": 0.208, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.1696798130869865, "train_loss_last": 4.462538719177246, "val_eval_loss": 5.125715062022209}
+{"condition": "probe_cal_blend_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.089, "dropout_final": 0.01, "dropout_initial": 0.208, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.222177028656, "eval_loss": 4.839077055454254, "generalization_gap": 0.7490236163139343, "model_config": {"block_size": 128, "dropout": 0.208, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.09005343914032, "train_loss_last": 4.155421257019043, "val_eval_loss": 4.839077055454254}
+{"condition": "probe_cal_blend_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.044, "dropout_final": 0.01, "dropout_initial": 0.208, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 101.6900281906128, "eval_loss": 4.649116978049278, "generalization_gap": 0.49115535616874695, "model_config": {"block_size": 128, "dropout": 0.208, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.157961621880531, "train_loss_last": 4.239448547363281, "val_eval_loss": 4.649116978049278}
+{"condition": "probe_cal_blend_local", "condition_kind": "anchor_decay", "dropout_active_final": 0.01, "dropout_final": 0.01, "dropout_initial": 0.208, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.13673806190491, "eval_loss": 4.502436943352222, "generalization_gap": 0.33670200407505035, "model_config": {"block_size": 128, "dropout": 0.208, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.165734939277172, "train_loss_last": 4.0054426193237305, "val_eval_loss": 4.502436943352222}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.73936200141907, "eval_loss": 5.495434820652008, "generalization_gap": 1.0909202694892883, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.40451455116272, "train_loss_last": 4.451443672180176, "val_eval_loss": 5.495434820652008}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.84255409240723, "eval_loss": 5.144781172275543, "generalization_gap": 1.053678423166275, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.091102749109268, "train_loss_last": 4.338654518127441, "val_eval_loss": 5.144781172275543}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.1, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.90904498100281, "eval_loss": 4.859739378094673, "generalization_gap": 0.7649403437972069, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.094799034297466, "train_loss_last": 4.2892608642578125, "val_eval_loss": 4.859739378094673}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.06, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.8770101070404, "eval_loss": 4.67372015863657, "generalization_gap": 0.48085135966539383, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.192868798971176, "train_loss_last": 4.079019546508789, "val_eval_loss": 4.67372015863657}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.84612607955933, "eval_loss": 4.506289839744568, "generalization_gap": 0.30511386692523956, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.201175972819328, "train_loss_last": 4.084702014923096, "val_eval_loss": 4.506289839744568}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.27022171020508, "eval_loss": 5.741428807377815, "generalization_gap": 1.7659062817692757, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.9755225256085396, "train_loss_last": 4.037149429321289, "val_eval_loss": 5.741428807377815}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.45338821411133, "eval_loss": 5.522661320865154, "generalization_gap": 1.9324281141161919, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.5902332067489624, "train_loss_last": 3.6823925971984863, "val_eval_loss": 5.522661320865154}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.44305801391602, "eval_loss": 5.13706586509943, "generalization_gap": 1.3302377983927727, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.8068280667066574, "train_loss_last": 3.807931900024414, "val_eval_loss": 5.13706586509943}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.355877161026, "eval_loss": 4.82523400336504, "generalization_gap": 0.745576985180378, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.079657018184662, "train_loss_last": 4.220707893371582, "val_eval_loss": 4.82523400336504}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.39533519744873, "eval_loss": 4.607164613902569, "generalization_gap": 0.4136744365096092, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.19349017739296, "train_loss_last": 4.070268630981445, "val_eval_loss": 4.607164613902569}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.27944421768188, "eval_loss": 5.628744050860405, "generalization_gap": 1.5491753369569778, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.079568713903427, "train_loss_last": 4.008928298950195, "val_eval_loss": 5.628744050860405}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.50746488571167, "eval_loss": 5.418625384569168, "generalization_gap": 1.7123678401112556, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.7062575444579124, "train_loss_last": 3.892080307006836, "val_eval_loss": 5.418625384569168}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.32422113418579, "eval_loss": 5.031492792069912, "generalization_gap": 1.137767605483532, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.89372518658638, "train_loss_last": 3.9074714183807373, "val_eval_loss": 5.031492792069912}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 102.97934222221375, "eval_loss": 4.7777098044753075, "generalization_gap": 0.6714786291122437, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.106231175363064, "train_loss_last": 4.278778553009033, "val_eval_loss": 4.7777098044753075}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 102.05160307884216, "eval_loss": 4.562453582882881, "generalization_gap": 0.349097803235054, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.213355779647827, "train_loss_last": 4.3637261390686035, "val_eval_loss": 4.562453582882881}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 102.82359099388123, "eval_loss": 5.607951521873474, "generalization_gap": 1.4676379710435867, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.140313550829887, "train_loss_last": 4.333317756652832, "val_eval_loss": 5.607951521873474}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.5397720336914, "eval_loss": 5.327187150716782, "generalization_gap": 1.5662295073270798, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.760957643389702, "train_loss_last": 3.914050579071045, "val_eval_loss": 5.327187150716782}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.58205389976501, "eval_loss": 4.967750117182732, "generalization_gap": 1.0176998153328896, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.950050301849842, "train_loss_last": 3.6760218143463135, "val_eval_loss": 4.967750117182732}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.53292489051819, "eval_loss": 4.727139674127102, "generalization_gap": 0.600127600133419, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.127012073993683, "train_loss_last": 4.172330856323242, "val_eval_loss": 4.727139674127102}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.45511293411255, "eval_loss": 4.539536610245705, "generalization_gap": 0.3323665261268616, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.207170084118843, "train_loss_last": 4.197327136993408, "val_eval_loss": 4.539536610245705}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 103.40926575660706, "eval_loss": 5.535304918885231, "generalization_gap": 1.2749911397695541, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.260313779115677, "train_loss_last": 4.383523941040039, "val_eval_loss": 5.535304918885231}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 102.78240418434143, "eval_loss": 5.23005124181509, "generalization_gap": 1.3044280409812927, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.9256232008337975, "train_loss_last": 4.204541206359863, "val_eval_loss": 5.23005124181509}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.89656591415405, "eval_loss": 4.900925785303116, "generalization_gap": 0.8582110702991486, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.042714715003967, "train_loss_last": 4.023549556732178, "val_eval_loss": 4.900925785303116}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.79082322120667, "eval_loss": 4.6937364637851715, "generalization_gap": 0.51191246509552, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.1818239986896515, "train_loss_last": 4.045550346374512, "val_eval_loss": 4.6937364637851715}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 103.2428879737854, "eval_loss": 4.527845799922943, "generalization_gap": 0.27384504675865173, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.254000753164291, "train_loss_last": 4.359174728393555, "val_eval_loss": 4.527845799922943}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 103.33639597892761, "eval_loss": 5.49543484300375, "generalization_gap": 1.0909202322363853, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.4045146107673645, "train_loss_last": 4.451443672180176, "val_eval_loss": 5.49543484300375}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 103.41859793663025, "eval_loss": 5.144781269133091, "generalization_gap": 1.0536784306168556, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.091102838516235, "train_loss_last": 4.338654518127441, "val_eval_loss": 5.144781269133091}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 103.53811812400818, "eval_loss": 4.8538225665688515, "generalization_gap": 0.703917995095253, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.1499045714735985, "train_loss_last": 4.3441691398620605, "val_eval_loss": 4.8538225665688515}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 103.53125715255737, "eval_loss": 4.667612560093403, "generalization_gap": 0.40461840480566025, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.262994155287743, "train_loss_last": 4.194393634796143, "val_eval_loss": 4.667612560093403}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 103.5893018245697, "eval_loss": 4.520538829267025, "generalization_gap": 0.23658638447523117, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.283952444791794, "train_loss_last": 4.277287006378174, "val_eval_loss": 4.520538829267025}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 103.45598697662354, "eval_loss": 5.485036075115204, "generalization_gap": 0.94332554936409, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.541710525751114, "train_loss_last": 4.596960067749023, "val_eval_loss": 5.485036075115204}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 103.19505786895752, "eval_loss": 5.134363099932671, "generalization_gap": 0.9186945632100105, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 4.21566853672266, "train_loss_last": 4.590941905975342, "val_eval_loss": 5.134363099932671}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 103.35330510139465, "eval_loss": 4.844109363853931, "generalization_gap": 0.5997102782130241, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 4.244399085640907, "train_loss_last": 4.672922134399414, "val_eval_loss": 4.844109363853931}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 102.39623188972473, "eval_loss": 4.673134744167328, "generalization_gap": 0.34993769228458405, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 4.323197051882744, "train_loss_last": 4.352773666381836, "val_eval_loss": 4.673134744167328}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 102.69739127159119, "eval_loss": 4.533929988741875, "generalization_gap": 0.2260781079530716, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 4.307851880788803, "train_loss_last": 4.4729695320129395, "val_eval_loss": 4.533929988741875}

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/summary.csv

@@ -0,0 +1,51 @@
+run_mode,condition,condition_kind,stage,token_limit,model_name,n_layer,n_head,n_embd,parameters,dropout_initial,dropout_final,dropout_schedule,n,mean_train_eval_loss,std_train_eval_loss,mean_val_eval_loss,std_val_eval_loss,mean_generalization_gap,std_generalization_gap
+locked_stream,low_decay_heuristic,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,4.40451455116272,0.0,5.495434820652008,0.0,1.0909202694892883,0.0
+locked_stream,old_formula_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.75082091987133,0.0,5.53056164085865,0.0,0.77974072098732,0.0
+locked_stream,probe_cal_blend_local,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.208,0.01,log_prefix_anchor,1,4.552169069647789,0.0,5.490625523030758,0.0,0.9384564533829689,0.0
+locked_stream,probe_cal_strict_local,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.115,0.0,log_prefix_anchor,1,4.36291791498661,0.0,5.510357916355133,0.0,1.1474400013685226,0.0
+locked_stream,static_dropout_0,static,0,250000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.9755225256085396,0.0,5.741428807377815,0.0,1.7659062817692757,0.0
+locked_stream,static_dropout_0.02,static,0,250000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,4.079568713903427,0.0,5.628744050860405,0.0,1.5491753369569778,0.0
+locked_stream,static_dropout_0.04,static,0,250000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,4.140313550829887,0.0,5.607951521873474,0.0,1.4676379710435867,0.0
+locked_stream,static_dropout_0.08,static,0,250000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,4.260313779115677,0.0,5.535304918885231,0.0,1.2749911397695541,0.0
+locked_stream,static_dropout_0.14,static,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,4.4045146107673645,0.0,5.49543484300375,0.0,1.0909202322363853,0.0
+locked_stream,static_dropout_0.2,static,0,250000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.541710525751114,0.0,5.485036075115204,0.0,0.94332554936409,0.0
+locked_stream,low_decay_heuristic,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,4.091102749109268,0.0,5.144781172275543,0.0,1.053678423166275,0.0
+locked_stream,old_formula_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.344457596540451,0.0,5.1356144696474075,0.0,0.7911568731069565,0.0
+locked_stream,probe_cal_blend_local,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.208,0.01,log_prefix_anchor,1,4.1696798130869865,0.0,5.125715062022209,0.0,0.9560352489352226,0.0
+locked_stream,probe_cal_strict_local,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.115,0.0,log_prefix_anchor,1,3.9278043285012245,0.0,5.221328005194664,0.0,1.2935236766934395,0.0
+locked_stream,static_dropout_0,static,1,500000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.5902332067489624,0.0,5.522661320865154,0.0,1.9324281141161919,0.0
+locked_stream,static_dropout_0.02,static,1,500000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.7062575444579124,0.0,5.418625384569168,0.0,1.7123678401112556,0.0
+locked_stream,static_dropout_0.04,static,1,500000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.760957643389702,0.0,5.327187150716782,0.0,1.5662295073270798,0.0
+locked_stream,static_dropout_0.08,static,1,500000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.9256232008337975,0.0,5.23005124181509,0.0,1.3044280409812927,0.0
+locked_stream,static_dropout_0.14,static,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,4.091102838516235,0.0,5.144781269133091,0.0,1.0536784306168556,0.0
+locked_stream,static_dropout_0.2,static,1,500000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.21566853672266,0.0,5.134363099932671,0.0,0.9186945632100105,0.0
+locked_stream,low_decay_heuristic,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,4.094799034297466,0.0,4.859739378094673,0.0,0.7649403437972069,0.0
+locked_stream,old_formula_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.249339297413826,0.0,4.834159925580025,0.0,0.5848206281661987,0.0
+locked_stream,probe_cal_blend_local,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.208,0.01,log_prefix_anchor,1,4.09005343914032,0.0,4.839077055454254,0.0,0.7490236163139343,0.0
+locked_stream,probe_cal_strict_local,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.115,0.0,log_prefix_anchor,1,3.901294879615307,0.0,5.0118745639920235,0.0,1.1105796843767166,0.0
+locked_stream,static_dropout_0,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.8068280667066574,0.0,5.13706586509943,0.0,1.3302377983927727,0.0
+locked_stream,static_dropout_0.02,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.89372518658638,0.0,5.031492792069912,0.0,1.137767605483532,0.0
+locked_stream,static_dropout_0.04,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.950050301849842,0.0,4.967750117182732,0.0,1.0176998153328896,0.0
+locked_stream,static_dropout_0.08,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,4.042714715003967,0.0,4.900925785303116,0.0,0.8582110702991486,0.0
+locked_stream,static_dropout_0.14,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,4.1499045714735985,0.0,4.8538225665688515,0.0,0.703917995095253,0.0
+locked_stream,static_dropout_0.2,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.244399085640907,0.0,4.844109363853931,0.0,0.5997102782130241,0.0
+locked_stream,low_decay_heuristic,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,4.192868798971176,0.0,4.67372015863657,0.0,0.48085135966539383,0.0
+locked_stream,old_formula_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.22222713381052,0.0,4.640849784016609,0.0,0.418622650206089,0.0
+locked_stream,probe_cal_blend_local,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.208,0.01,log_prefix_anchor,1,4.157961621880531,0.0,4.649116978049278,0.0,0.49115535616874695,0.0
+locked_stream,probe_cal_strict_local,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.115,0.0,log_prefix_anchor,1,4.0890104323625565,0.0,4.763735756278038,0.0,0.6747253239154816,0.0
+locked_stream,static_dropout_0,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,4.079657018184662,0.0,4.82523400336504,0.0,0.745576985180378,0.0
+locked_stream,static_dropout_0.02,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,4.106231175363064,0.0,4.7777098044753075,0.0,0.6714786291122437,0.0
+locked_stream,static_dropout_0.04,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,4.127012073993683,0.0,4.727139674127102,0.0,0.600127600133419,0.0
+locked_stream,static_dropout_0.08,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,4.1818239986896515,0.0,4.6937364637851715,0.0,0.51191246509552,0.0
+locked_stream,static_dropout_0.14,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,4.262994155287743,0.0,4.667612560093403,0.0,0.40461840480566025,0.0
+locked_stream,static_dropout_0.2,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.323197051882744,0.0,4.673134744167328,0.0,0.34993769228458405,0.0
+locked_stream,low_decay_heuristic,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,4.201175972819328,0.0,4.506289839744568,0.0,0.30511386692523956,0.0
+locked_stream,old_formula_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.217783153057098,0.0,4.483362518250942,0.0,0.26557936519384384,0.0
+locked_stream,probe_cal_blend_local,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.208,0.01,log_prefix_anchor,1,4.165734939277172,0.0,4.502436943352222,0.0,0.33670200407505035,0.0
+locked_stream,probe_cal_strict_local,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.115,0.0,log_prefix_anchor,1,4.181766301393509,0.0,4.560461081564426,0.0,0.3786947801709175,0.0
+locked_stream,static_dropout_0,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,4.19349017739296,0.0,4.607164613902569,0.0,0.4136744365096092,0.0
+locked_stream,static_dropout_0.02,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,4.213355779647827,0.0,4.562453582882881,0.0,0.349097803235054,0.0
+locked_stream,static_dropout_0.04,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,4.207170084118843,0.0,4.539536610245705,0.0,0.3323665261268616,0.0
+locked_stream,static_dropout_0.08,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,4.254000753164291,0.0,4.527845799922943,0.0,0.27384504675865173,0.0
+locked_stream,static_dropout_0.14,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,4.283952444791794,0.0,4.520538829267025,0.0,0.23658638447523117,0.0
+locked_stream,static_dropout_0.2,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.307851880788803,0.0,4.533929988741875,0.0,0.2260781079530716,0.0

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/summary.json

@@ -0,0 +1,1102 @@
+[
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.40451455116272,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.495434820652008,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.0909202694892883,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.75082091987133,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.53056164085865,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.77974072098732,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_local",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.208,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.552169069647789,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.490625523030758,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9384564533829689,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_local",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.115,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.36291791498661,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.510357916355133,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1474400013685226,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.9755225256085396,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.741428807377815,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.7659062817692757,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.079568713903427,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.628744050860405,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.5491753369569778,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.140313550829887,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.607951521873474,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.4676379710435867,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.260313779115677,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.535304918885231,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.2749911397695541,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.4045146107673645,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.49543484300375,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.0909202322363853,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.541710525751114,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.485036075115204,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.94332554936409,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.091102749109268,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.144781172275543,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.053678423166275,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.344457596540451,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.1356144696474075,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7911568731069565,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_local",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.208,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.1696798130869865,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.125715062022209,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9560352489352226,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_local",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.115,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.9278043285012245,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.221328005194664,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.2935236766934395,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.5902332067489624,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.522661320865154,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.9324281141161919,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.7062575444579124,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.418625384569168,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.7123678401112556,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.760957643389702,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.327187150716782,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.5662295073270798,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.9256232008337975,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.23005124181509,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.3044280409812927,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.091102838516235,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.144781269133091,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.0536784306168556,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.21566853672266,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.134363099932671,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9186945632100105,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.094799034297466,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.859739378094673,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7649403437972069,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.249339297413826,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.834159925580025,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5848206281661987,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_local",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.208,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.09005343914032,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.839077055454254,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7490236163139343,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_local",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.115,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.901294879615307,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.0118745639920235,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1105796843767166,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8068280667066574,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.13706586509943,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.3302377983927727,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.89372518658638,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.031492792069912,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.137767605483532,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.950050301849842,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.967750117182732,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.0176998153328896,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.042714715003967,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.900925785303116,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8582110702991486,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.1499045714735985,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.8538225665688515,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.703917995095253,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.244399085640907,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.844109363853931,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5997102782130241,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.192868798971176,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.67372015863657,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.48085135966539383,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.22222713381052,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.640849784016609,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.418622650206089,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_local",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.208,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.157961621880531,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.649116978049278,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.49115535616874695,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_local",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.115,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.0890104323625565,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.763735756278038,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6747253239154816,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.079657018184662,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.82523400336504,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.745576985180378,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.106231175363064,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.7777098044753075,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6714786291122437,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.127012073993683,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.727139674127102,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.600127600133419,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.1818239986896515,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.6937364637851715,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.51191246509552,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.262994155287743,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.667612560093403,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.40461840480566025,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.323197051882744,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.673134744167328,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.34993769228458405,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.201175972819328,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.506289839744568,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.30511386692523956,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.217783153057098,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.483362518250942,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.26557936519384384,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_local",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.208,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.165734939277172,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.502436943352222,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.33670200407505035,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_local",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.115,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.181766301393509,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.560461081564426,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3786947801709175,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.19349017739296,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.607164613902569,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4136744365096092,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.213355779647827,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.562453582882881,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.349097803235054,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.207170084118843,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.539536610245705,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3323665261268616,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.254000753164291,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.527845799922943,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.27384504675865173,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.283952444791794,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.520538829267025,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.23658638447523117,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.307851880788803,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.533929988741875,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2260781079530716,
+    "std_generalization_gap": 0.0
+  }
+]

runs/probe_calibrated_stream_local/locked_stream/20260529-093748/trace.jsonl

@@ -0,0 +1,100 @@
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.354827880859375}
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.885682582855225}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 5.032894134521484}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.61283540725708}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.5941667556762695}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.361461639404297}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.511430740356445}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.314380645751953}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.49089241027832}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.028076171875}
+{"condition": "probe_cal_strict_local", "dropout": 0.115, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.284113883972168}
+{"condition": "probe_cal_strict_local", "dropout": 0.115, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.475851535797119}
+{"condition": "probe_cal_strict_local", "dropout": 0.069, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.288115501403809}
+{"condition": "probe_cal_strict_local", "dropout": 0.069, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.225583076477051}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.399591445922852}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.032376289367676}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.137774467468262}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.111223220825195}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.1803741455078125}
+{"condition": "probe_cal_strict_local", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.159558296203613}
+{"condition": "probe_cal_blend_local", "dropout": 0.208, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.403038024902344}
+{"condition": "probe_cal_blend_local", "dropout": 0.208, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.479777812957764}
+{"condition": "probe_cal_blend_local", "dropout": 0.162, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.654302597045898}
+{"condition": "probe_cal_blend_local", "dropout": 0.162, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.462538719177246}
+{"condition": "probe_cal_blend_local", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.507874488830566}
+{"condition": "probe_cal_blend_local", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.155421257019043}
+{"condition": "probe_cal_blend_local", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.174427032470703}
+{"condition": "probe_cal_blend_local", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.239448547363281}
+{"condition": "probe_cal_blend_local", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.157096862792969}
+{"condition": "probe_cal_blend_local", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.0054426193237305}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.256674766540527}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.451443672180176}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.67259407043457}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.338654518127441}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.484482765197754}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.2892608642578125}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.290275573730469}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.079019546508789}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.202578067779541}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.084702014923096}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.1745500564575195}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.037149429321289}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.131137847900391}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.6823925971984863}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.042705059051514}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.807931900024414}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.390109062194824}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.220707893371582}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.172452449798584}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.070268630981445}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.130340576171875}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.008928298950195}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.144742012023926}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.892080307006836}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.962122917175293}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.9074714183807373}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.2829132080078125}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.278778553009033}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.520293712615967}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.3637261390686035}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.077821254730225}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.333317756652832}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.311123847961426}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.914050579071045}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.499200820922852}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.6760218143463135}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.285571098327637}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.172330856323242}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.4568634033203125}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.197327136993408}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.3072052001953125}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.383523941040039}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.1660308837890625}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.204541206359863}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.235754013061523}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.023549556732178}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.227766513824463}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.045550346374512}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.341914176940918}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.359174728393555}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.256674766540527}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.451443672180176}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.67259407043457}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.338654518127441}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.54632568359375}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.3441691398620605}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.39534854888916}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.194393634796143}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.330488681793213}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.277287006378174}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 5.230059623718262}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.596960067749023}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.666769027709961}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 4.590941905975342}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.503047943115234}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 4.672922134399414}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.425793647766113}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 4.352773666381836}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.478578567504883}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.4729695320129395}

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/RESULT_SUMMARY.md

@@ -0,0 +1,98 @@
+# Locked Streaming Dropout Summary
+
+Run directory: `runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913`
+
+Model: `L12_H8_D320` causal Transformer, 17,367,040 parameters, 12 layers, 8 heads, 320 embedding dim.
+Training per stage: 1,000 steps. Sampled tokens are cumulative in each stage row. Seeds present: 1.
+
+## Condition Ranking
+
+| Condition | Kind | Final dropout | Mean trajectory val loss | Final val loss | Final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---|
+| `low_decay_heuristic` | anchor_decay | 0.02 | 3.0793 | 2.7068 | 0.2161 | 0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02 |
+| `static_dropout_0.08` | static | 0.08 | 3.0880 | 2.7215 | 0.2060 | 0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08 |
+| `static_dropout_0.14` | static | 0.14 | 3.0891 | 2.7353 | 0.1793 | 0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14 |
+| `probe_cal_blend_tinystories` | anchor_decay | 0.01 | 3.0934 | 2.7194 | 0.2275 | 0.20 -> 0.15 -> 0.09 -> 0.04 -> 0.01 |
+| `static_dropout_0.2` | static | 0.20 | 3.1095 | 2.7681 | 0.1747 | 0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20 |
+| `old_formula_l12` | anchor_decay | 0.02 | 3.1149 | 2.7221 | 0.2033 | 0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02 |
+| `static_dropout_0.04` | static | 0.04 | 3.1175 | 2.7293 | 0.2324 | 0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04 |
+| `probe_cal_strict_tinystories` | anchor_decay | 0.00 | 3.1178 | 2.7312 | 0.2626 | 0.10 -> 0.05 -> 0.00 -> 0.00 -> 0.00 |
+| `static_dropout_0.02` | static | 0.02 | 3.1394 | 2.7309 | 0.2481 | 0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02 |
+| `static_dropout_0` | static | 0.00 | 3.2153 | 2.7720 | 0.2950 | 0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00 |
+
+## Stage Trajectory
+
+### Stage 0: 250,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `low_decay_heuristic` | 0.14 | 3.6243 | 3.0415 | 0.5828 | 1 |
+| `static_dropout_0.14` | 0.14 | 3.6243 | 3.0415 | 0.5828 | 1 |
+| `static_dropout_0.08` | 0.08 | 3.6258 | 2.9575 | 0.6683 | 1 |
+| `probe_cal_strict_tinystories` | 0.10 | 3.6264 | 2.9728 | 0.6536 | 1 |
+| `static_dropout_0.2` | 0.20 | 3.6435 | 3.1453 | 0.4982 | 1 |
+| `static_dropout_0.04` | 0.04 | 3.6501 | 2.8501 | 0.8000 | 1 |
+| `probe_cal_blend_tinystories` | 0.20 | 3.6542 | 3.1430 | 0.5112 | 1 |
+| `static_dropout_0.02` | 0.02 | 3.6543 | 2.8092 | 0.8451 | 1 |
+| `old_formula_l12` | 0.30 | 3.7006 | 3.2967 | 0.4039 | 1 |
+| `static_dropout_0` | 0.00 | 3.7198 | 2.7344 | 0.9855 | 1 |
+
+### Stage 1: 500,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `static_dropout_0.14` | 0.14 | 3.2501 | 2.7564 | 0.4937 | 1 |
+| `low_decay_heuristic` | 0.14 | 3.2501 | 2.7564 | 0.4937 | 1 |
+| `static_dropout_0.08` | 0.08 | 3.2570 | 2.6329 | 0.6241 | 1 |
+| `static_dropout_0.2` | 0.20 | 3.2625 | 2.8462 | 0.4163 | 1 |
+| `probe_cal_blend_tinystories` | 0.15 | 3.2630 | 2.8096 | 0.4534 | 1 |
+| `probe_cal_strict_tinystories` | 0.05 | 3.2834 | 2.5820 | 0.7014 | 1 |
+| `old_formula_l12` | 0.26 | 3.2920 | 2.9384 | 0.3537 | 1 |
+| `static_dropout_0.04` | 0.04 | 3.3114 | 2.5082 | 0.8032 | 1 |
+| `static_dropout_0.02` | 0.02 | 3.3677 | 2.4883 | 0.8794 | 1 |
+| `static_dropout_0` | 0.00 | 3.4876 | 2.3957 | 1.0920 | 1 |
+
+### Stage 2: 1,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `low_decay_heuristic` | 0.10 | 2.9938 | 2.6547 | 0.3391 | 1 |
+| `static_dropout_0.14` | 0.14 | 2.9994 | 2.6922 | 0.3072 | 1 |
+| `probe_cal_blend_tinystories` | 0.09 | 3.0047 | 2.6670 | 0.3378 | 1 |
+| `static_dropout_0.2` | 0.20 | 3.0114 | 2.7754 | 0.2360 | 1 |
+| `static_dropout_0.08` | 0.08 | 3.0139 | 2.6214 | 0.3925 | 1 |
+| `old_formula_l12` | 0.18 | 3.0237 | 2.7853 | 0.2384 | 1 |
+| `static_dropout_0.04` | 0.04 | 3.0487 | 2.5580 | 0.4907 | 1 |
+| `probe_cal_strict_tinystories` | 0.00 | 3.0724 | 2.5159 | 0.5566 | 1 |
+| `static_dropout_0.02` | 0.02 | 3.0728 | 2.5107 | 0.5621 | 1 |
+| `static_dropout_0` | 0.00 | 3.1697 | 2.4597 | 0.7099 | 1 |
+
+### Stage 3: 2,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `low_decay_heuristic` | 0.06 | 2.8217 | 2.5694 | 0.2523 | 1 |
+| `static_dropout_0.08` | 0.08 | 2.8220 | 2.5529 | 0.2691 | 1 |
+| `probe_cal_blend_tinystories` | 0.04 | 2.8255 | 2.5643 | 0.2611 | 1 |
+| `old_formula_l12` | 0.09 | 2.8359 | 2.6234 | 0.2125 | 1 |
+| `static_dropout_0.14` | 0.14 | 2.8365 | 2.6240 | 0.2125 | 1 |
+| `static_dropout_0.04` | 0.04 | 2.8481 | 2.5228 | 0.3252 | 1 |
+| `static_dropout_0.2` | 0.20 | 2.8617 | 2.6824 | 0.1793 | 1 |
+| `static_dropout_0.02` | 0.02 | 2.8712 | 2.5029 | 0.3683 | 1 |
+| `probe_cal_strict_tinystories` | 0.00 | 2.8754 | 2.4844 | 0.3910 | 1 |
+| `static_dropout_0` | 0.00 | 2.9274 | 2.4931 | 0.4343 | 1 |
+
+### Stage 4: 4,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `low_decay_heuristic` | 0.02 | 2.7068 | 2.4907 | 0.2161 | 1 |
+| `probe_cal_blend_tinystories` | 0.01 | 2.7194 | 2.4919 | 0.2275 | 1 |
+| `static_dropout_0.08` | 0.08 | 2.7215 | 2.5155 | 0.2060 | 1 |
+| `old_formula_l12` | 0.02 | 2.7221 | 2.5187 | 0.2033 | 1 |
+| `static_dropout_0.04` | 0.04 | 2.7293 | 2.4970 | 0.2324 | 1 |
+| `static_dropout_0.02` | 0.02 | 2.7309 | 2.4828 | 0.2481 | 1 |
+| `probe_cal_strict_tinystories` | 0.00 | 2.7312 | 2.4685 | 0.2626 | 1 |
+| `static_dropout_0.14` | 0.14 | 2.7353 | 2.5560 | 0.1793 | 1 |
+| `static_dropout_0.2` | 0.20 | 2.7681 | 2.5935 | 0.1747 | 1 |
+| `static_dropout_0` | 0.00 | 2.7720 | 2.4770 | 0.2950 | 1 |

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/config.json

@@ -0,0 +1,216 @@
+{
+  "args": {
+    "mode": "locked_stream",
+    "corpus": null,
+    "corpus_glob": null,
+    "text_column": "text",
+    "use_cached_data": true,
+    "output_dir": "runs/probe_calibrated_stream_tinystories",
+    "resume_from": null,
+    "cache_dir": ".cache/dropout_decay_tinystories",
+    "models": [
+      "L12_H8_D320=12x8x320"
+    ],
+    "seeds": [
+      1
+    ],
+    "token_limits": [
+      5000000
+    ],
+    "stream_token_caps": [
+      250000,
+      500000,
+      1000000,
+      2000000,
+      4000000
+    ],
+    "val_tokens": 500000,
+    "allow_short_corpus": false,
+    "force_retokenize": false,
+    "vocab_size": 4096,
+    "tokenizer_train_chars": 10000000,
+    "block_size": 128,
+    "batch_size": 16,
+    "steps": 2000,
+    "stage_steps": 1000,
+    "dropout_rates": [
+      0.0,
+      0.02,
+      0.04,
+      0.08,
+      0.14,
+      0.2
+    ],
+    "decays": [],
+    "anchor_decays": [
+      {
+        "name": "old_formula_l12",
+        "kind": "anchor_decay",
+        "initial": 0.301,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.301
+          ],
+          [
+            500000,
+            0.255
+          ],
+          [
+            1000000,
+            0.177
+          ],
+          [
+            2000000,
+            0.087
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_strict_tinystories",
+        "kind": "anchor_decay",
+        "initial": 0.095,
+        "final": 0.0,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.095
+          ],
+          [
+            500000,
+            0.049
+          ],
+          [
+            1000000,
+            0.0
+          ],
+          [
+            2000000,
+            0.0
+          ],
+          [
+            4000000,
+            0.0
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_blend_tinystories",
+        "kind": "anchor_decay",
+        "initial": 0.198,
+        "final": 0.01,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.198
+          ],
+          [
+            500000,
+            0.152
+          ],
+          [
+            1000000,
+            0.089
+          ],
+          [
+            2000000,
+            0.044
+          ],
+          [
+            4000000,
+            0.01
+          ]
+        ]
+      },
+      {
+        "name": "low_decay_heuristic",
+        "kind": "anchor_decay",
+        "initial": 0.14,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.14
+          ],
+          [
+            500000,
+            0.14
+          ],
+          [
+            1000000,
+            0.1
+          ],
+          [
+            2000000,
+            0.06
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      }
+    ],
+    "decay_tokens": null,
+    "eval_batches": 64,
+    "train_eval_batches": 32,
+    "trace_eval_batches": 8,
+    "eval_every": 0,
+    "log_every": 500,
+    "lr": 0.0003,
+    "weight_decay": 0.1,
+    "grad_clip": 1.0,
+    "plateau_delta": 0.01,
+    "target_min_dropout": 0.1,
+    "min_nonzero_margin": 0.01,
+    "min_high_dropout_margin": 0.03,
+    "screen_early_stop": false,
+    "screen_prune_patience": 3,
+    "screen_prune_min_delta": 0.01
+  },
+  "mode": "locked_stream",
+  "seeds": [
+    1
+  ],
+  "models": [
+    {
+      "model_name": "L12_H8_D320",
+      "n_layer": 12,
+      "n_head": 8,
+      "n_embd": 320
+    }
+  ],
+  "device": "mps",
+  "torch": "2.12.0",
+  "python": "3.11.15 (main, Mar  3 2026, 00:52:57) [Clang 21.0.0 (clang-2100.0.123.102)]",
+  "mps_available": true,
+  "attribution": "Derived from Andrej Karpathy's nanochat project (https://github.com/karpathy/nanochat), MIT License, Copyright (c) 2025 Andrej Karpathy.",
+  "tokenizer_path": ".cache/dropout_decay_tinystories/tokenizer-v4096.json",
+  "encoded_path": ".cache/dropout_decay_tinystories/tokens-v4096-uint16.npy",
+  "train_tokens": 4500048,
+  "val_tokens": 500000,
+  "effective_token_limits": [
+    4500048
+  ],
+  "effective_stream_token_caps": [
+    250000,
+    500000,
+    1000000,
+    2000000,
+    4000000
+  ],
+  "resume_from": null
+}

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/metrics.jsonl

@@ -0,0 +1,50 @@
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.301, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.62233591079712, "eval_loss": 3.700591169297695, "generalization_gap": 0.4038587063550949, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.2967324629426003, "train_loss_last": 3.4338440895080566, "val_eval_loss": 3.700591169297695}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.255, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.884202003479, "eval_loss": 3.2920290417969227, "generalization_gap": 0.3536711297929287, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.938357912003994, "train_loss_last": 3.019982099533081, "val_eval_loss": 3.2920290417969227}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.177, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 101.9442150592804, "eval_loss": 3.0236747823655605, "generalization_gap": 0.2384135015308857, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.785261280834675, "train_loss_last": 2.8230879306793213, "val_eval_loss": 3.0236747823655605}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.087, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.0156888961792, "eval_loss": 2.8359112292528152, "generalization_gap": 0.21254538744688034, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.623365841805935, "train_loss_last": 2.5181002616882324, "val_eval_loss": 2.8359112292528152}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.66161108016968, "eval_loss": 2.7220530919730663, "generalization_gap": 0.20334095880389214, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.518712133169174, "train_loss_last": 2.521733283996582, "val_eval_loss": 2.7220530919730663}
+{"condition": "probe_cal_strict_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.095, "dropout_final": 0.0, "dropout_initial": 0.095, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.46083092689514, "eval_loss": 3.6264022290706635, "generalization_gap": 0.6535925343632698, "model_config": {"block_size": 128, "dropout": 0.095, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 2.9728096947073936, "train_loss_last": 3.0214738845825195, "val_eval_loss": 3.6264022290706635}
+{"condition": "probe_cal_strict_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.049, "dropout_final": 0.0, "dropout_initial": 0.095, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.49551391601562, "eval_loss": 3.2834053114056587, "generalization_gap": 0.7014138028025627, "model_config": {"block_size": 128, "dropout": 0.095, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.581991508603096, "train_loss_last": 2.589329481124878, "val_eval_loss": 3.2834053114056587}
+{"condition": "probe_cal_strict_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.095, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.92377591133118, "eval_loss": 3.0724188946187496, "generalization_gap": 0.5565654672682285, "model_config": {"block_size": 128, "dropout": 0.095, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.515853427350521, "train_loss_last": 2.4549384117126465, "val_eval_loss": 3.0724188946187496}
+{"condition": "probe_cal_strict_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.095, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 99.37469387054443, "eval_loss": 2.8754161782562733, "generalization_gap": 0.3910205252468586, "model_config": {"block_size": 128, "dropout": 0.095, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.4843956530094147, "train_loss_last": 2.6597843170166016, "val_eval_loss": 2.8754161782562733}
+{"condition": "probe_cal_strict_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.095, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 98.99975800514221, "eval_loss": 2.731159035116434, "generalization_gap": 0.26264239475131035, "model_config": {"block_size": 128, "dropout": 0.095, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.4685166403651237, "train_loss_last": 2.3526194095611572, "val_eval_loss": 2.731159035116434}
+{"condition": "probe_cal_blend_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.198, "dropout_final": 0.01, "dropout_initial": 0.198, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.29193711280823, "eval_loss": 3.6542300321161747, "generalization_gap": 0.5112252496182919, "model_config": {"block_size": 128, "dropout": 0.198, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.143004782497883, "train_loss_last": 3.258655548095703, "val_eval_loss": 3.6542300321161747}
+{"condition": "probe_cal_blend_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.152, "dropout_final": 0.01, "dropout_initial": 0.198, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.25700092315674, "eval_loss": 3.263014566153288, "generalization_gap": 0.45343584194779396, "model_config": {"block_size": 128, "dropout": 0.198, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.809578724205494, "train_loss_last": 2.9130990505218506, "val_eval_loss": 3.263014566153288}
+{"condition": "probe_cal_blend_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.089, "dropout_final": 0.01, "dropout_initial": 0.198, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.29276990890503, "eval_loss": 3.0047357082366943, "generalization_gap": 0.3377818614244461, "model_config": {"block_size": 128, "dropout": 0.198, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.6669538468122482, "train_loss_last": 3.0272746086120605, "val_eval_loss": 3.0047357082366943}
+{"condition": "probe_cal_blend_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.044, "dropout_final": 0.01, "dropout_initial": 0.198, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 102.79022693634033, "eval_loss": 2.8254924304783344, "generalization_gap": 0.2611430399119854, "model_config": {"block_size": 128, "dropout": 0.198, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.564349390566349, "train_loss_last": 2.5292603969573975, "val_eval_loss": 2.8254924304783344}
+{"condition": "probe_cal_blend_tinystories", "condition_kind": "anchor_decay", "dropout_active_final": 0.01, "dropout_final": 0.01, "dropout_initial": 0.198, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.0759527683258, "eval_loss": 2.719374742358923, "generalization_gap": 0.22748106345534325, "model_config": {"block_size": 128, "dropout": 0.198, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.4918936789035797, "train_loss_last": 2.6417012214660645, "val_eval_loss": 2.719374742358923}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.97193598747253, "eval_loss": 3.624329824000597, "generalization_gap": 0.582804050296545, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.041525773704052, "train_loss_last": 3.2490952014923096, "val_eval_loss": 3.624329824000597}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.24307584762573, "eval_loss": 3.250116005539894, "generalization_gap": 0.4937126860022545, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.7564033195376396, "train_loss_last": 2.782670497894287, "val_eval_loss": 3.250116005539894}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.1, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.02099013328552, "eval_loss": 2.9938040003180504, "generalization_gap": 0.3390728235244751, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.6547311767935753, "train_loss_last": 2.808277130126953, "val_eval_loss": 2.9938040003180504}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.06, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.56162405014038, "eval_loss": 2.8216672353446484, "generalization_gap": 0.2523050867021084, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.56936214864254, "train_loss_last": 2.579594612121582, "val_eval_loss": 2.8216672353446484}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.36827802658081, "eval_loss": 2.7067598327994347, "generalization_gap": 0.21610477566719055, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.490655057132244, "train_loss_last": 2.5801124572753906, "val_eval_loss": 2.7067598327994347}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.86895084381104, "eval_loss": 3.719829112291336, "generalization_gap": 0.9854638502001762, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 2.73436526209116, "train_loss_last": 2.720512866973877, "val_eval_loss": 3.719829112291336}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.8092930316925, "eval_loss": 3.4876452423632145, "generalization_gap": 1.0919925533235073, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.395652689039707, "train_loss_last": 2.310917854309082, "val_eval_loss": 3.4876452423632145}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.89768314361572, "eval_loss": 3.1696526780724525, "generalization_gap": 0.7099494189023972, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.4597032591700554, "train_loss_last": 2.5153374671936035, "val_eval_loss": 3.1696526780724525}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 99.30670714378357, "eval_loss": 2.9273829609155655, "generalization_gap": 0.43427007645368576, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.4931128844618797, "train_loss_last": 2.7157742977142334, "val_eval_loss": 2.9273829609155655}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 98.30992007255554, "eval_loss": 2.7719735391438007, "generalization_gap": 0.29496047273278236, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.4770130664110184, "train_loss_last": 2.344344139099121, "val_eval_loss": 2.7719735391438007}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 101.59399604797363, "eval_loss": 3.654279440641403, "generalization_gap": 0.8450979366898537, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 2.8091815039515495, "train_loss_last": 2.8237950801849365, "val_eval_loss": 3.654279440641403}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 102.98673415184021, "eval_loss": 3.3677087612450123, "generalization_gap": 0.8793939910829067, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.4883147701621056, "train_loss_last": 2.4535675048828125, "val_eval_loss": 3.3677087612450123}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.41584777832031, "eval_loss": 3.0728346183896065, "generalization_gap": 0.5621254593133926, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.510709159076214, "train_loss_last": 2.5469930171966553, "val_eval_loss": 3.0728346183896065}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.41725897789001, "eval_loss": 2.8711985126137733, "generalization_gap": 0.36832985281944275, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.5028686597943306, "train_loss_last": 2.6125242710113525, "val_eval_loss": 2.8711985126137733}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.35282492637634, "eval_loss": 2.730887234210968, "generalization_gap": 0.2480642944574356, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.4828229397535324, "train_loss_last": 2.5636167526245117, "val_eval_loss": 2.730887234210968}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.39403700828552, "eval_loss": 3.6500764675438404, "generalization_gap": 0.8000105284154415, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 2.850065939128399, "train_loss_last": 2.961707592010498, "val_eval_loss": 3.6500764675438404}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.50085091590881, "eval_loss": 3.3113764114677906, "generalization_gap": 0.803168524056673, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.5082078874111176, "train_loss_last": 2.736527919769287, "val_eval_loss": 3.3113764114677906}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.4635739326477, "eval_loss": 3.0486728698015213, "generalization_gap": 0.4907216355204582, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.557951234281063, "train_loss_last": 2.535484552383423, "val_eval_loss": 3.0486728698015213}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 103.2720160484314, "eval_loss": 2.848052367568016, "generalization_gap": 0.32523027062416077, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.5228220969438553, "train_loss_last": 2.3365275859832764, "val_eval_loss": 2.848052367568016}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 102.47527194023132, "eval_loss": 2.729311775416136, "generalization_gap": 0.2323615811765194, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.4969501942396164, "train_loss_last": 2.569467067718506, "val_eval_loss": 2.729311775416136}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.44355487823486, "eval_loss": 3.6257686764001846, "generalization_gap": 0.6682634726166725, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 2.957505203783512, "train_loss_last": 3.2516915798187256, "val_eval_loss": 3.6257686764001846}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.41104674339294, "eval_loss": 3.257015246897936, "generalization_gap": 0.6241430155932903, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.6328722313046455, "train_loss_last": 2.6162405014038086, "val_eval_loss": 3.257015246897936}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.49879193305969, "eval_loss": 3.013891890645027, "generalization_gap": 0.3924613371491432, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.621430553495884, "train_loss_last": 2.6491830348968506, "val_eval_loss": 3.013891890645027}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.5458550453186, "eval_loss": 2.8219608068466187, "generalization_gap": 0.26906655728816986, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.552894249558449, "train_loss_last": 2.6810357570648193, "val_eval_loss": 2.8219608068466187}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 101.64880800247192, "eval_loss": 2.7214900217950344, "generalization_gap": 0.20597973838448524, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.515510283410549, "train_loss_last": 2.610279083251953, "val_eval_loss": 2.7214900217950344}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 101.79736971855164, "eval_loss": 3.624329876154661, "generalization_gap": 0.5828041471540928, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.0415257290005684, "train_loss_last": 3.2490952014923096, "val_eval_loss": 3.624329876154661}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 101.97021007537842, "eval_loss": 3.250115979462862, "generalization_gap": 0.4937126822769642, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.756403297185898, "train_loss_last": 2.782670497894287, "val_eval_loss": 3.250115979462862}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 102.06508493423462, "eval_loss": 2.999407649040222, "generalization_gap": 0.30722276121377945, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.6921848878264427, "train_loss_last": 2.8426907062530518, "val_eval_loss": 2.999407649040222}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 101.9444010257721, "eval_loss": 2.8365167379379272, "generalization_gap": 0.2125382348895073, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.62397850304842, "train_loss_last": 2.668032169342041, "val_eval_loss": 2.8365167379379272}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 101.81766128540039, "eval_loss": 2.7352735362946987, "generalization_gap": 0.17931919172406197, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.5559543445706367, "train_loss_last": 2.7375197410583496, "val_eval_loss": 2.7352735362946987}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.79993891716003, "eval_loss": 3.6435023061931133, "generalization_gap": 0.4981841780245304, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.145318128168583, "train_loss_last": 3.326073169708252, "val_eval_loss": 3.6435023061931133}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.89059090614319, "eval_loss": 3.2625456377863884, "generalization_gap": 0.4163428992033005, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 2.846202738583088, "train_loss_last": 2.9462132453918457, "val_eval_loss": 3.2625456377863884}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.88215517997742, "eval_loss": 3.0113844461739063, "generalization_gap": 0.23597873374819756, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 2.7754057124257088, "train_loss_last": 3.0238137245178223, "val_eval_loss": 3.0113844461739063}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.94479894638062, "eval_loss": 2.86172816157341, "generalization_gap": 0.1793275848031044, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 2.6824005767703056, "train_loss_last": 2.6997509002685547, "val_eval_loss": 2.86172816157341}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 102.26193308830261, "eval_loss": 2.7681099846959114, "generalization_gap": 0.1746557652950287, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 2.5934542194008827, "train_loss_last": 2.6558125019073486, "val_eval_loss": 2.7681099846959114}

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/summary.csv

@@ -0,0 +1,51 @@
+run_mode,condition,condition_kind,stage,token_limit,model_name,n_layer,n_head,n_embd,parameters,dropout_initial,dropout_final,dropout_schedule,n,mean_train_eval_loss,std_train_eval_loss,mean_val_eval_loss,std_val_eval_loss,mean_generalization_gap,std_generalization_gap
+locked_stream,low_decay_heuristic,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.041525773704052,0.0,3.624329824000597,0.0,0.582804050296545,0.0
+locked_stream,old_formula_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,3.2967324629426003,0.0,3.700591169297695,0.0,0.4038587063550949,0.0
+locked_stream,probe_cal_blend_tinystories,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.198,0.01,log_prefix_anchor,1,3.143004782497883,0.0,3.6542300321161747,0.0,0.5112252496182919,0.0
+locked_stream,probe_cal_strict_tinystories,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.095,0.0,log_prefix_anchor,1,2.9728096947073936,0.0,3.6264022290706635,0.0,0.6535925343632698,0.0
+locked_stream,static_dropout_0,static,0,250000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,2.73436526209116,0.0,3.719829112291336,0.0,0.9854638502001762,0.0
+locked_stream,static_dropout_0.02,static,0,250000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,2.8091815039515495,0.0,3.654279440641403,0.0,0.8450979366898537,0.0
+locked_stream,static_dropout_0.04,static,0,250000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,2.850065939128399,0.0,3.6500764675438404,0.0,0.8000105284154415,0.0
+locked_stream,static_dropout_0.08,static,0,250000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,2.957505203783512,0.0,3.6257686764001846,0.0,0.6682634726166725,0.0
+locked_stream,static_dropout_0.14,static,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.0415257290005684,0.0,3.624329876154661,0.0,0.5828041471540928,0.0
+locked_stream,static_dropout_0.2,static,0,250000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,3.145318128168583,0.0,3.6435023061931133,0.0,0.4981841780245304,0.0
+locked_stream,low_decay_heuristic,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,2.7564033195376396,0.0,3.250116005539894,0.0,0.4937126860022545,0.0
+locked_stream,old_formula_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,2.938357912003994,0.0,3.2920290417969227,0.0,0.3536711297929287,0.0
+locked_stream,probe_cal_blend_tinystories,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.198,0.01,log_prefix_anchor,1,2.809578724205494,0.0,3.263014566153288,0.0,0.45343584194779396,0.0
+locked_stream,probe_cal_strict_tinystories,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.095,0.0,log_prefix_anchor,1,2.581991508603096,0.0,3.2834053114056587,0.0,0.7014138028025627,0.0
+locked_stream,static_dropout_0,static,1,500000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,2.395652689039707,0.0,3.4876452423632145,0.0,1.0919925533235073,0.0
+locked_stream,static_dropout_0.02,static,1,500000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,2.4883147701621056,0.0,3.3677087612450123,0.0,0.8793939910829067,0.0
+locked_stream,static_dropout_0.04,static,1,500000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,2.5082078874111176,0.0,3.3113764114677906,0.0,0.803168524056673,0.0
+locked_stream,static_dropout_0.08,static,1,500000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,2.6328722313046455,0.0,3.257015246897936,0.0,0.6241430155932903,0.0
+locked_stream,static_dropout_0.14,static,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,2.756403297185898,0.0,3.250115979462862,0.0,0.4937126822769642,0.0
+locked_stream,static_dropout_0.2,static,1,500000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,2.846202738583088,0.0,3.2625456377863884,0.0,0.4163428992033005,0.0
+locked_stream,low_decay_heuristic,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,2.6547311767935753,0.0,2.9938040003180504,0.0,0.3390728235244751,0.0
+locked_stream,old_formula_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,2.785261280834675,0.0,3.0236747823655605,0.0,0.2384135015308857,0.0
+locked_stream,probe_cal_blend_tinystories,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.198,0.01,log_prefix_anchor,1,2.6669538468122482,0.0,3.0047357082366943,0.0,0.3377818614244461,0.0
+locked_stream,probe_cal_strict_tinystories,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.095,0.0,log_prefix_anchor,1,2.515853427350521,0.0,3.0724188946187496,0.0,0.5565654672682285,0.0
+locked_stream,static_dropout_0,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,2.4597032591700554,0.0,3.1696526780724525,0.0,0.7099494189023972,0.0
+locked_stream,static_dropout_0.02,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,2.510709159076214,0.0,3.0728346183896065,0.0,0.5621254593133926,0.0
+locked_stream,static_dropout_0.04,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,2.557951234281063,0.0,3.0486728698015213,0.0,0.4907216355204582,0.0
+locked_stream,static_dropout_0.08,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,2.621430553495884,0.0,3.013891890645027,0.0,0.3924613371491432,0.0
+locked_stream,static_dropout_0.14,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,2.6921848878264427,0.0,2.999407649040222,0.0,0.30722276121377945,0.0
+locked_stream,static_dropout_0.2,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,2.7754057124257088,0.0,3.0113844461739063,0.0,0.23597873374819756,0.0
+locked_stream,low_decay_heuristic,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,2.56936214864254,0.0,2.8216672353446484,0.0,0.2523050867021084,0.0
+locked_stream,old_formula_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,2.623365841805935,0.0,2.8359112292528152,0.0,0.21254538744688034,0.0
+locked_stream,probe_cal_blend_tinystories,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.198,0.01,log_prefix_anchor,1,2.564349390566349,0.0,2.8254924304783344,0.0,0.2611430399119854,0.0
+locked_stream,probe_cal_strict_tinystories,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.095,0.0,log_prefix_anchor,1,2.4843956530094147,0.0,2.8754161782562733,0.0,0.3910205252468586,0.0
+locked_stream,static_dropout_0,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,2.4931128844618797,0.0,2.9273829609155655,0.0,0.43427007645368576,0.0
+locked_stream,static_dropout_0.02,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,2.5028686597943306,0.0,2.8711985126137733,0.0,0.36832985281944275,0.0
+locked_stream,static_dropout_0.04,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,2.5228220969438553,0.0,2.848052367568016,0.0,0.32523027062416077,0.0
+locked_stream,static_dropout_0.08,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,2.552894249558449,0.0,2.8219608068466187,0.0,0.26906655728816986,0.0
+locked_stream,static_dropout_0.14,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,2.62397850304842,0.0,2.8365167379379272,0.0,0.2125382348895073,0.0
+locked_stream,static_dropout_0.2,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,2.6824005767703056,0.0,2.86172816157341,0.0,0.1793275848031044,0.0
+locked_stream,low_decay_heuristic,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,2.490655057132244,0.0,2.7067598327994347,0.0,0.21610477566719055,0.0
+locked_stream,old_formula_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,2.518712133169174,0.0,2.7220530919730663,0.0,0.20334095880389214,0.0
+locked_stream,probe_cal_blend_tinystories,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.198,0.01,log_prefix_anchor,1,2.4918936789035797,0.0,2.719374742358923,0.0,0.22748106345534325,0.0
+locked_stream,probe_cal_strict_tinystories,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.095,0.0,log_prefix_anchor,1,2.4685166403651237,0.0,2.731159035116434,0.0,0.26264239475131035,0.0
+locked_stream,static_dropout_0,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,2.4770130664110184,0.0,2.7719735391438007,0.0,0.29496047273278236,0.0
+locked_stream,static_dropout_0.02,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,2.4828229397535324,0.0,2.730887234210968,0.0,0.2480642944574356,0.0
+locked_stream,static_dropout_0.04,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,2.4969501942396164,0.0,2.729311775416136,0.0,0.2323615811765194,0.0
+locked_stream,static_dropout_0.08,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,2.515510283410549,0.0,2.7214900217950344,0.0,0.20597973838448524,0.0
+locked_stream,static_dropout_0.14,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,2.5559543445706367,0.0,2.7352735362946987,0.0,0.17931919172406197,0.0
+locked_stream,static_dropout_0.2,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,2.5934542194008827,0.0,2.7681099846959114,0.0,0.1746557652950287,0.0

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/summary.json

@@ -0,0 +1,1102 @@
+[
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.041525773704052,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.624329824000597,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.582804050296545,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.2967324629426003,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.700591169297695,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4038587063550949,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.198,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.143004782497883,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.6542300321161747,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5112252496182919,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.095,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.9728096947073936,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.6264022290706635,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6535925343632698,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.73436526209116,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.719829112291336,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9854638502001762,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.8091815039515495,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.654279440641403,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8450979366898537,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.850065939128399,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.6500764675438404,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8000105284154415,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.957505203783512,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.6257686764001846,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6682634726166725,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.0415257290005684,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.624329876154661,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5828041471540928,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.145318128168583,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.6435023061931133,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4981841780245304,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.7564033195376396,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.250116005539894,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4937126860022545,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.938357912003994,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.2920290417969227,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3536711297929287,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.198,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.809578724205494,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.263014566153288,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.45343584194779396,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.095,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.581991508603096,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.2834053114056587,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7014138028025627,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.395652689039707,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.4876452423632145,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.0919925533235073,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4883147701621056,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.3677087612450123,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8793939910829067,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.5082078874111176,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.3113764114677906,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.803168524056673,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.6328722313046455,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.257015246897936,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6241430155932903,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.756403297185898,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.250115979462862,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4937126822769642,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.846202738583088,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.2625456377863884,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4163428992033005,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.6547311767935753,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.9938040003180504,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3390728235244751,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.785261280834675,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0236747823655605,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2384135015308857,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.198,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.6669538468122482,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0047357082366943,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3377818614244461,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.095,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.515853427350521,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0724188946187496,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5565654672682285,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4597032591700554,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.1696526780724525,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7099494189023972,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.510709159076214,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0728346183896065,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5621254593133926,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.557951234281063,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0486728698015213,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4907216355204582,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.621430553495884,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.013891890645027,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3924613371491432,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.6921848878264427,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.999407649040222,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.30722276121377945,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.7754057124257088,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 3.0113844461739063,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.23597873374819756,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.56936214864254,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8216672353446484,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2523050867021084,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.623365841805935,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8359112292528152,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.21254538744688034,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.198,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.564349390566349,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8254924304783344,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2611430399119854,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.095,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.4843956530094147,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8754161782562733,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3910205252468586,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4931128844618797,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.9273829609155655,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.43427007645368576,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.5028686597943306,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8711985126137733,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.36832985281944275,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.5228220969438553,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.848052367568016,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.32523027062416077,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.552894249558449,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8219608068466187,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.26906655728816986,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.62397850304842,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.8365167379379272,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2125382348895073,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.6824005767703056,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.86172816157341,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.1793275848031044,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.490655057132244,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7067598327994347,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.21610477566719055,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.518712133169174,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7220530919730663,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.20334095880389214,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.198,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.4918936789035797,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.719374742358923,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.22748106345534325,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_tinystories",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.095,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 2.4685166403651237,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.731159035116434,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.26264239475131035,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4770130664110184,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7719735391438007,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.29496047273278236,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4828229397535324,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.730887234210968,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2480642944574356,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.4969501942396164,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.729311775416136,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2323615811765194,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.515510283410549,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7214900217950344,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.20597973838448524,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.5559543445706367,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7352735362946987,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.17931919172406197,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 2.5934542194008827,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 2.7681099846959114,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.1746557652950287,
+    "std_generalization_gap": 0.0
+  }
+]

runs/probe_calibrated_stream_tinystories/locked_stream/20260529-122913/trace.jsonl

@@ -0,0 +1,100 @@
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.009261131286621}
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.4338440895080566}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.1904613971710205}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.019982099533081}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.10953950881958}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.8230879306793213}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.57950496673584}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.5181002616882324}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.4837703704833984}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.521733283996582}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.095, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.53718900680542}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.095, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.0214738845825195}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.049, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 2.8355624675750732}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.049, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.589329481124878}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.766968250274658}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.4549384117126465}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.822331666946411}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.6597843170166016}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.7321925163269043}
+{"condition": "probe_cal_strict_tinystories", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.3526194095611572}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.198, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.6930785179138184}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.198, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.258655548095703}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.152, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.0700674057006836}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.152, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.9130990505218506}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.7304203510284424}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.0272746086120605}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.686694622039795}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.5292603969573975}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.570572853088379}
+{"condition": "probe_cal_blend_tinystories", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.6417012214660645}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.9024057388305664}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.2490952014923096}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.078380584716797}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.782670497894287}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.883002758026123}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.808277130126953}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.8963921070098877}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.579594612121582}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.597691297531128}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.5801124572753906}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.629361152648926}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 2.720512866973877}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 2.9673118591308594}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.310917854309082}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.6466121673583984}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.5153374671936035}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.482654333114624}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.7157742977142334}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.5386781692504883}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.344344139099121}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.71156644821167}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 2.8237950801849365}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 2.608339309692383}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.4535675048828125}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.709308385848999}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.5469930171966553}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.801713466644287}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.6125242710113525}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.52340030670166}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.5636167526245117}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.5526983737945557}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 2.961707592010498}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 2.8655471801757812}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.736527919769287}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.7962021827697754}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.535484552383423}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.7107763290405273}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.3365275859832764}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.4338138103485107}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.569467067718506}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.5524544715881348}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.2516915798187256}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 2.7981762886047363}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.6162405014038086}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.724429130554199}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.6491830348968506}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.772150993347168}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.6810357570648193}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.8927083015441895}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.610279083251953}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.9024057388305664}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.2490952014923096}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.078380584716797}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.782670497894287}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 2.9295413494110107}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 2.8426907062530518}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.9844841957092285}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.668032169342041}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.7342376708984375}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.7375197410583496}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 3.630584716796875}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.326073169708252}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.190708875656128}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 2.9462132453918457}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.065217971801758}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.0238137245178223}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 2.7958874702453613}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 2.6997509002685547}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 2.7621688842773438}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 2.6558125019073486}

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/RESULT_SUMMARY.md

@@ -0,0 +1,98 @@
+# Locked Streaming Dropout Summary
+
+Run directory: `runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250`
+
+Model: `L12_H8_D320` causal Transformer, 17,367,040 parameters, 12 layers, 8 heads, 320 embedding dim.
+Training per stage: 1,000 steps. Sampled tokens are cumulative in each stage row. Seeds present: 1.
+
+## Condition Ranking
+
+| Condition | Kind | Final dropout | Mean trajectory val loss | Final val loss | Final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---|
+| `probe_cal_blend_wikitext103` | anchor_decay | 0.01 | 4.5557 | 4.0689 | 0.2981 | 0.19 -> 0.14 -> 0.09 -> 0.04 -> 0.01 |
+| `low_decay_heuristic` | anchor_decay | 0.02 | 4.5683 | 4.0854 | 0.2930 | 0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02 |
+| `old_formula_l12` | anchor_decay | 0.02 | 4.5704 | 4.0643 | 0.2430 | 0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02 |
+| `static_dropout_0.2` | static | 0.20 | 4.5727 | 4.1162 | 0.1831 | 0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20 |
+| `static_dropout_0.14` | static | 0.14 | 4.5736 | 4.1082 | 0.2231 | 0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14 |
+| `static_dropout_0.08` | static | 0.08 | 4.5889 | 4.0893 | 0.2449 | 0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08 |
+| `static_dropout_0.04` | static | 0.04 | 4.6407 | 4.1031 | 0.2896 | 0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04 |
+| `probe_cal_strict_wikitext103` | anchor_decay | 0.00 | 4.6759 | 4.1441 | 0.3769 | 0.07 -> 0.03 -> 0.00 -> 0.00 -> 0.00 |
+| `static_dropout_0.02` | static | 0.02 | 4.7021 | 4.1371 | 0.3320 | 0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02 |
+| `static_dropout_0` | static | 0.00 | 4.7804 | 4.1600 | 0.3894 | 0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00 |
+
+## Stage Trajectory
+
+### Stage 0: 250,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `static_dropout_0.2` | 0.20 | 5.1763 | 4.0387 | 1.1376 | 1 |
+| `probe_cal_blend_wikitext103` | 0.19 | 5.1764 | 4.0190 | 1.1574 | 1 |
+| `low_decay_heuristic` | 0.14 | 5.1792 | 3.8959 | 1.2833 | 1 |
+| `static_dropout_0.14` | 0.14 | 5.1792 | 3.8959 | 1.2833 | 1 |
+| `static_dropout_0.08` | 0.08 | 5.1941 | 3.7549 | 1.4392 | 1 |
+| `probe_cal_strict_wikitext103` | 0.07 | 5.2057 | 3.7475 | 1.4582 | 1 |
+| `old_formula_l12` | 0.30 | 5.2323 | 4.2327 | 0.9996 | 1 |
+| `static_dropout_0.04` | 0.04 | 5.2336 | 3.6665 | 1.5672 | 1 |
+| `static_dropout_0.02` | 0.02 | 5.2866 | 3.5968 | 1.6898 | 1 |
+| `static_dropout_0` | 0.00 | 5.3558 | 3.4670 | 1.8888 | 1 |
+
+### Stage 1: 500,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `static_dropout_0.2` | 0.20 | 4.7516 | 3.7633 | 0.9883 | 1 |
+| `probe_cal_blend_wikitext103` | 0.14 | 4.7555 | 3.6517 | 1.1038 | 1 |
+| `old_formula_l12` | 0.26 | 4.7667 | 3.8876 | 0.8791 | 1 |
+| `low_decay_heuristic` | 0.14 | 4.7802 | 3.6468 | 1.1333 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.7802 | 3.6468 | 1.1333 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.8277 | 3.4745 | 1.3532 | 1 |
+| `probe_cal_strict_wikitext103` | 0.03 | 4.9212 | 3.3205 | 1.6007 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.9274 | 3.3200 | 1.6074 | 1 |
+| `static_dropout_0.02` | 0.02 | 5.0339 | 3.2596 | 1.7743 | 1 |
+| `static_dropout_0` | 0.00 | 5.1645 | 3.1207 | 2.0438 | 1 |
+
+### Stage 2: 1,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `probe_cal_blend_wikitext103` | 0.09 | 4.4902 | 3.6384 | 0.8518 | 1 |
+| `old_formula_l12` | 0.18 | 4.5063 | 3.8260 | 0.6803 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.5074 | 3.7114 | 0.7960 | 1 |
+| `low_decay_heuristic` | 0.10 | 4.5087 | 3.6537 | 0.8550 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.5108 | 3.8256 | 0.6852 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.5318 | 3.5962 | 0.9356 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.6007 | 3.5105 | 1.0902 | 1 |
+| `static_dropout_0.02` | 0.02 | 4.6654 | 3.4419 | 1.2235 | 1 |
+| `probe_cal_strict_wikitext103` | 0.00 | 4.7060 | 3.3731 | 1.3329 | 1 |
+| `static_dropout_0` | 0.00 | 4.7937 | 3.3347 | 1.4590 | 1 |
+
+### Stage 3: 2,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `old_formula_l12` | 0.09 | 4.2826 | 3.8000 | 0.4826 | 1 |
+| `probe_cal_blend_wikitext103` | 0.04 | 4.2874 | 3.7199 | 0.5675 | 1 |
+| `low_decay_heuristic` | 0.06 | 4.2880 | 3.7483 | 0.5397 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.2932 | 3.8263 | 0.4669 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.3016 | 3.7278 | 0.5739 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.3088 | 3.8944 | 0.4144 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.3388 | 3.6748 | 0.6639 | 1 |
+| `static_dropout_0.02` | 0.02 | 4.3877 | 3.6570 | 0.7307 | 1 |
+| `probe_cal_strict_wikitext103` | 0.00 | 4.4023 | 3.6160 | 0.7863 | 1 |
+| `static_dropout_0` | 0.00 | 4.4281 | 3.6167 | 0.8114 | 1 |
+
+### Stage 4: 4,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `old_formula_l12` | 0.02 | 4.0643 | 3.8212 | 0.2430 | 1 |
+| `probe_cal_blend_wikitext103` | 0.01 | 4.0689 | 3.7708 | 0.2981 | 1 |
+| `low_decay_heuristic` | 0.02 | 4.0854 | 3.7924 | 0.2930 | 1 |
+| `static_dropout_0.08` | 0.08 | 4.0893 | 3.8443 | 0.2449 | 1 |
+| `static_dropout_0.04` | 0.04 | 4.1031 | 3.8135 | 0.2896 | 1 |
+| `static_dropout_0.14` | 0.14 | 4.1082 | 3.8851 | 0.2231 | 1 |
+| `static_dropout_0.2` | 0.20 | 4.1162 | 3.9331 | 0.1831 | 1 |
+| `static_dropout_0.02` | 0.02 | 4.1371 | 3.8051 | 0.3320 | 1 |
+| `probe_cal_strict_wikitext103` | 0.00 | 4.1441 | 3.7672 | 0.3769 | 1 |
+| `static_dropout_0` | 0.00 | 4.1600 | 3.7706 | 0.3894 | 1 |

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/config.json

@@ -0,0 +1,216 @@
+{
+  "args": {
+    "mode": "locked_stream",
+    "corpus": null,
+    "corpus_glob": null,
+    "text_column": "text",
+    "use_cached_data": true,
+    "output_dir": "runs/probe_calibrated_stream_wikitext103",
+    "resume_from": null,
+    "cache_dir": ".cache/dropout_decay_wikitext103",
+    "models": [
+      "L12_H8_D320=12x8x320"
+    ],
+    "seeds": [
+      1
+    ],
+    "token_limits": [
+      5000000
+    ],
+    "stream_token_caps": [
+      250000,
+      500000,
+      1000000,
+      2000000,
+      4000000
+    ],
+    "val_tokens": 500000,
+    "allow_short_corpus": false,
+    "force_retokenize": false,
+    "vocab_size": 4096,
+    "tokenizer_train_chars": 10000000,
+    "block_size": 128,
+    "batch_size": 16,
+    "steps": 2000,
+    "stage_steps": 1000,
+    "dropout_rates": [
+      0.0,
+      0.02,
+      0.04,
+      0.08,
+      0.14,
+      0.2
+    ],
+    "decays": [],
+    "anchor_decays": [
+      {
+        "name": "old_formula_l12",
+        "kind": "anchor_decay",
+        "initial": 0.301,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.301
+          ],
+          [
+            500000,
+            0.255
+          ],
+          [
+            1000000,
+            0.177
+          ],
+          [
+            2000000,
+            0.087
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_strict_wikitext103",
+        "kind": "anchor_decay",
+        "initial": 0.075,
+        "final": 0.0,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.075
+          ],
+          [
+            500000,
+            0.029
+          ],
+          [
+            1000000,
+            0.0
+          ],
+          [
+            2000000,
+            0.0
+          ],
+          [
+            4000000,
+            0.0
+          ]
+        ]
+      },
+      {
+        "name": "probe_cal_blend_wikitext103",
+        "kind": "anchor_decay",
+        "initial": 0.188,
+        "final": 0.01,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.188
+          ],
+          [
+            500000,
+            0.142
+          ],
+          [
+            1000000,
+            0.089
+          ],
+          [
+            2000000,
+            0.044
+          ],
+          [
+            4000000,
+            0.01
+          ]
+        ]
+      },
+      {
+        "name": "low_decay_heuristic",
+        "kind": "anchor_decay",
+        "initial": 0.14,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.14
+          ],
+          [
+            500000,
+            0.14
+          ],
+          [
+            1000000,
+            0.1
+          ],
+          [
+            2000000,
+            0.06
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      }
+    ],
+    "decay_tokens": null,
+    "eval_batches": 64,
+    "train_eval_batches": 32,
+    "trace_eval_batches": 8,
+    "eval_every": 0,
+    "log_every": 500,
+    "lr": 0.0003,
+    "weight_decay": 0.1,
+    "grad_clip": 1.0,
+    "plateau_delta": 0.01,
+    "target_min_dropout": 0.1,
+    "min_nonzero_margin": 0.01,
+    "min_high_dropout_margin": 0.03,
+    "screen_early_stop": false,
+    "screen_prune_patience": 3,
+    "screen_prune_min_delta": 0.01
+  },
+  "mode": "locked_stream",
+  "seeds": [
+    1
+  ],
+  "models": [
+    {
+      "model_name": "L12_H8_D320",
+      "n_layer": 12,
+      "n_head": 8,
+      "n_embd": 320
+    }
+  ],
+  "device": "mps",
+  "torch": "2.12.0",
+  "python": "3.11.15 (main, Mar  3 2026, 00:52:57) [Clang 21.0.0 (clang-2100.0.123.102)]",
+  "mps_available": true,
+  "attribution": "Derived from Andrej Karpathy's nanochat project (https://github.com/karpathy/nanochat), MIT License, Copyright (c) 2025 Andrej Karpathy.",
+  "tokenizer_path": ".cache/dropout_decay_wikitext103/tokenizer-v4096.json",
+  "encoded_path": ".cache/dropout_decay_wikitext103/tokens-v4096-uint16.npy",
+  "train_tokens": 4500020,
+  "val_tokens": 500000,
+  "effective_token_limits": [
+    4500020
+  ],
+  "effective_stream_token_caps": [
+    250000,
+    500000,
+    1000000,
+    2000000,
+    4000000
+  ],
+  "resume_from": null
+}

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/metrics.jsonl

@@ -0,0 +1,50 @@
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.301, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 89.9664671421051, "eval_loss": 5.232291169464588, "generalization_gap": 0.9996200874447823, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.232671082019806, "train_loss_last": 4.400547981262207, "val_eval_loss": 5.232291169464588}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.255, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 93.61638307571411, "eval_loss": 4.7666677832603455, "generalization_gap": 0.8791004717350006, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.887567311525345, "train_loss_last": 3.966970443725586, "val_eval_loss": 4.7666677832603455}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.177, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 101.72808599472046, "eval_loss": 4.5063412338495255, "generalization_gap": 0.6803312972187996, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.826009936630726, "train_loss_last": 3.897003412246704, "val_eval_loss": 4.5063412338495255}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.087, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 107.66677618026733, "eval_loss": 4.282629065215588, "generalization_gap": 0.4826100841164589, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.8000189810991287, "train_loss_last": 3.769676923751831, "val_eval_loss": 4.282629065215588}
+{"condition": "old_formula_l12", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.301, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 107.46920990943909, "eval_loss": 4.064255967736244, "generalization_gap": 0.24302349239587784, "model_config": {"block_size": 128, "dropout": 0.301, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.8212324753403664, "train_loss_last": 3.9641218185424805, "val_eval_loss": 4.064255967736244}
+{"condition": "probe_cal_strict_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.075, "dropout_final": 0.0, "dropout_initial": 0.075, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 106.11210513114929, "eval_loss": 5.205722317099571, "generalization_gap": 1.4582452103495598, "model_config": {"block_size": 128, "dropout": 0.075, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.7474771067500114, "train_loss_last": 3.656510591506958, "val_eval_loss": 5.205722317099571}
+{"condition": "probe_cal_strict_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.028999999999999998, "dropout_final": 0.0, "dropout_initial": 0.075, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 105.34217023849487, "eval_loss": 4.92121696472168, "generalization_gap": 1.600672908127308, "model_config": {"block_size": 128, "dropout": 0.075, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.320544056594372, "train_loss_last": 3.5648581981658936, "val_eval_loss": 4.92121696472168}
+{"condition": "probe_cal_strict_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.075, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 101.38782811164856, "eval_loss": 4.705984242260456, "generalization_gap": 1.332908719778061, "model_config": {"block_size": 128, "dropout": 0.075, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.373075522482395, "train_loss_last": 3.1880264282226562, "val_eval_loss": 4.705984242260456}
+{"condition": "probe_cal_strict_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.075, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 100.87962603569031, "eval_loss": 4.402302749454975, "generalization_gap": 0.7862669825553894, "model_config": {"block_size": 128, "dropout": 0.075, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.6160357668995857, "train_loss_last": 3.779501438140869, "val_eval_loss": 4.402302749454975}
+{"condition": "probe_cal_strict_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.075, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 100.4942238330841, "eval_loss": 4.144088886678219, "generalization_gap": 0.37687263637781143, "model_config": {"block_size": 128, "dropout": 0.075, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.7672162503004074, "train_loss_last": 3.632549524307251, "val_eval_loss": 4.144088886678219}
+{"condition": "probe_cal_blend_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.188, "dropout_final": 0.01, "dropout_initial": 0.188, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.0358030796051, "eval_loss": 5.176444113254547, "generalization_gap": 1.1574340909719467, "model_config": {"block_size": 128, "dropout": 0.188, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.0190100222826, "train_loss_last": 4.025996208190918, "val_eval_loss": 5.176444113254547}
+{"condition": "probe_cal_blend_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.142, "dropout_final": 0.01, "dropout_initial": 0.188, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.30060410499573, "eval_loss": 4.755539648234844, "generalization_gap": 1.10379970818758, "model_config": {"block_size": 128, "dropout": 0.188, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.651739940047264, "train_loss_last": 3.9967105388641357, "val_eval_loss": 4.755539648234844}
+{"condition": "probe_cal_blend_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.089, "dropout_final": 0.01, "dropout_initial": 0.188, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.15587496757507, "eval_loss": 4.490205697715282, "generalization_gap": 0.8518205657601357, "model_config": {"block_size": 128, "dropout": 0.188, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.638385131955147, "train_loss_last": 3.895059823989868, "val_eval_loss": 4.490205697715282}
+{"condition": "probe_cal_blend_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.044, "dropout_final": 0.01, "dropout_initial": 0.188, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.97929000854492, "eval_loss": 4.287429004907608, "generalization_gap": 0.5674877241253853, "model_config": {"block_size": 128, "dropout": 0.188, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.7199412807822227, "train_loss_last": 3.9355649948120117, "val_eval_loss": 4.287429004907608}
+{"condition": "probe_cal_blend_wikitext103", "condition_kind": "anchor_decay", "dropout_active_final": 0.01, "dropout_final": 0.01, "dropout_initial": 0.188, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.81685662269592, "eval_loss": 4.068912364542484, "generalization_gap": 0.2981444299221039, "model_config": {"block_size": 128, "dropout": 0.188, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.7707679346203804, "train_loss_last": 3.897859811782837, "val_eval_loss": 4.068912364542484}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 103.99191689491272, "eval_loss": 5.179202765226364, "generalization_gap": 1.283341646194458, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.895861119031906, "train_loss_last": 3.8898677825927734, "val_eval_loss": 5.179202765226364}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.14, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.18846297264099, "eval_loss": 4.78017570823431, "generalization_gap": 1.1333301067352295, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.6468456014990807, "train_loss_last": 3.756572723388672, "val_eval_loss": 4.78017570823431}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.1, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.19513416290283, "eval_loss": 4.508680582046509, "generalization_gap": 0.8550295457243919, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.653651036322117, "train_loss_last": 3.709557056427002, "val_eval_loss": 4.508680582046509}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.06, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.0916359424591, "eval_loss": 4.287984453141689, "generalization_gap": 0.539727158844471, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.7482572942972183, "train_loss_last": 3.707911252975464, "val_eval_loss": 4.287984453141689}
+{"condition": "low_decay_heuristic", "condition_kind": "anchor_decay", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.14, "dropout_schedule": "log_prefix_anchor", "elapsed_sec": 104.05507922172546, "eval_loss": 4.085419524461031, "generalization_gap": 0.29301564767956734, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.7924038767814636, "train_loss_last": 3.6913557052612305, "val_eval_loss": 4.085419524461031}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 100.43107891082764, "eval_loss": 5.355821952223778, "generalization_gap": 1.8888395354151726, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.466982416808605, "train_loss_last": 3.4980170726776123, "val_eval_loss": 5.355821952223778}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 100.42404198646545, "eval_loss": 5.164503566920757, "generalization_gap": 2.0438041985034943, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.120699368417263, "train_loss_last": 3.010756015777588, "val_eval_loss": 5.164503566920757}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 100.46810507774353, "eval_loss": 4.793723180890083, "generalization_gap": 1.4590169042348862, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.334706276655197, "train_loss_last": 3.2843174934387207, "val_eval_loss": 4.793723180890083}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 100.57566690444946, "eval_loss": 4.428123250603676, "generalization_gap": 0.8114311099052429, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.616692140698433, "train_loss_last": 3.6820335388183594, "val_eval_loss": 4.428123250603676}
+{"condition": "static_dropout_0", "condition_kind": "static", "dropout_active_final": 0.0, "dropout_final": 0.0, "dropout_initial": 0.0, "dropout_schedule": "constant", "elapsed_sec": 100.47721481323242, "eval_loss": 4.159992642700672, "generalization_gap": 0.3893583118915558, "model_config": {"block_size": 128, "dropout": 0.0, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.7706343308091164, "train_loss_last": 3.5455589294433594, "val_eval_loss": 4.159992642700672}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 104.07356023788452, "eval_loss": 5.286626905202866, "generalization_gap": 1.6898214146494865, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.596805490553379, "train_loss_last": 3.5398495197296143, "val_eval_loss": 5.286626905202866}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 104.23886489868164, "eval_loss": 5.033853515982628, "generalization_gap": 1.7742735967040062, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.2595799192786217, "train_loss_last": 3.1747989654541016, "val_eval_loss": 5.033853515982628}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 104.114590883255, "eval_loss": 4.665390767157078, "generalization_gap": 1.2235022336244583, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.4418885335326195, "train_loss_last": 3.477747917175293, "val_eval_loss": 4.665390767157078}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 104.07853317260742, "eval_loss": 4.387734830379486, "generalization_gap": 0.7307379841804504, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.6569968461990356, "train_loss_last": 3.790534496307373, "val_eval_loss": 4.387734830379486}
+{"condition": "static_dropout_0.02", "condition_kind": "static", "dropout_active_final": 0.02, "dropout_final": 0.02, "dropout_initial": 0.02, "dropout_schedule": "constant", "elapsed_sec": 103.97844195365906, "eval_loss": 4.137094311416149, "generalization_gap": 0.3319554775953293, "model_config": {"block_size": 128, "dropout": 0.02, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.80513883382082, "train_loss_last": 3.661900281906128, "val_eval_loss": 4.137094311416149}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 104.18495678901672, "eval_loss": 5.233644910156727, "generalization_gap": 1.567173771560192, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.6664711385965347, "train_loss_last": 3.7955689430236816, "val_eval_loss": 5.233644910156727}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 104.22118282318115, "eval_loss": 4.927406623959541, "generalization_gap": 1.6073753014206886, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.3200313225388527, "train_loss_last": 3.7024903297424316, "val_eval_loss": 4.927406623959541}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 104.09509587287903, "eval_loss": 4.600722283124924, "generalization_gap": 1.090194471180439, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.5105278119444847, "train_loss_last": 3.3805878162384033, "val_eval_loss": 4.600722283124924}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 104.16041612625122, "eval_loss": 4.338778018951416, "generalization_gap": 0.6639280617237091, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.674849957227707, "train_loss_last": 3.938136577606201, "val_eval_loss": 4.338778018951416}
+{"condition": "static_dropout_0.04", "condition_kind": "static", "dropout_active_final": 0.04, "dropout_final": 0.04, "dropout_initial": 0.04, "dropout_schedule": "constant", "elapsed_sec": 104.3692979812622, "eval_loss": 4.103101808577776, "generalization_gap": 0.2896107845008373, "model_config": {"block_size": 128, "dropout": 0.04, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.8134910240769386, "train_loss_last": 3.9043874740600586, "val_eval_loss": 4.103101808577776}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 104.07579970359802, "eval_loss": 5.194120541214943, "generalization_gap": 1.4391892403364182, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.754931300878525, "train_loss_last": 4.008996963500977, "val_eval_loss": 5.194120541214943}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 104.17510032653809, "eval_loss": 4.827699206769466, "generalization_gap": 1.3532356396317482, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.474463567137718, "train_loss_last": 3.538743495941162, "val_eval_loss": 4.827699206769466}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 104.33514380455017, "eval_loss": 4.531755141913891, "generalization_gap": 0.9355539679527283, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.5962011739611626, "train_loss_last": 3.636242389678955, "val_eval_loss": 4.531755141913891}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 104.31049609184265, "eval_loss": 4.301640450954437, "generalization_gap": 0.5738605707883835, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.7277798801660538, "train_loss_last": 3.6634957790374756, "val_eval_loss": 4.301640450954437}
+{"condition": "static_dropout_0.08", "condition_kind": "static", "dropout_active_final": 0.08, "dropout_final": 0.08, "dropout_initial": 0.08, "dropout_schedule": "constant", "elapsed_sec": 104.27281713485718, "eval_loss": 4.089278068393469, "generalization_gap": 0.24494213238358498, "model_config": {"block_size": 128, "dropout": 0.08, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.844335936009884, "train_loss_last": 3.8889031410217285, "val_eval_loss": 4.089278068393469}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 104.37981724739075, "eval_loss": 5.179202765226364, "generalization_gap": 1.2833416312932968, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 3.8958611339330673, "train_loss_last": 3.8898675441741943, "val_eval_loss": 5.179202765226364}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 104.2717559337616, "eval_loss": 4.780175752937794, "generalization_gap": 1.1333301290869713, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.6468456238508224, "train_loss_last": 3.75657320022583, "val_eval_loss": 4.780175752937794}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 104.32464981079102, "eval_loss": 4.507372200489044, "generalization_gap": 0.7959605753421783, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.711411625146866, "train_loss_last": 3.773858070373535, "val_eval_loss": 4.507372200489044}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 104.33226490020752, "eval_loss": 4.2931585647165775, "generalization_gap": 0.4668935351073742, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.8262650296092033, "train_loss_last": 3.8487484455108643, "val_eval_loss": 4.2931585647165775}
+{"condition": "static_dropout_0.14", "condition_kind": "static", "dropout_active_final": 0.14, "dropout_final": 0.14, "dropout_initial": 0.14, "dropout_schedule": "constant", "elapsed_sec": 104.05365204811096, "eval_loss": 4.1082155816257, "generalization_gap": 0.22314108535647392, "model_config": {"block_size": 128, "dropout": 0.14, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.885074496269226, "train_loss_last": 3.8854050636291504, "val_eval_loss": 4.1082155816257}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 103.82192420959473, "eval_loss": 5.176292642951012, "generalization_gap": 1.1375640854239464, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 0, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_eval_loss": 4.038728557527065, "train_loss_last": 4.0768561363220215, "val_eval_loss": 5.176292642951012}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 103.02905511856079, "eval_loss": 4.751599572598934, "generalization_gap": 0.9882605373859406, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 1, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_eval_loss": 3.7633390352129936, "train_loss_last": 3.8637547492980957, "val_eval_loss": 4.751599572598934}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.98273086547852, "eval_loss": 4.5107810869812965, "generalization_gap": 0.685192197561264, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 2, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_eval_loss": 3.8255888894200325, "train_loss_last": 3.981992483139038, "val_eval_loss": 4.5107810869812965}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.73532700538635, "eval_loss": 4.308795563876629, "generalization_gap": 0.41435327380895615, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 3, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_eval_loss": 3.8944422900676727, "train_loss_last": 3.994189500808716, "val_eval_loss": 4.308795563876629}
+{"condition": "static_dropout_0.2", "condition_kind": "static", "dropout_active_final": 0.2, "dropout_final": 0.2, "dropout_initial": 0.2, "dropout_schedule": "constant", "elapsed_sec": 101.89507007598877, "eval_loss": 4.1162367314100266, "generalization_gap": 0.1831037700176239, "model_config": {"block_size": 128, "dropout": 0.2, "n_embd": 320, "n_head": 8, "n_layer": 12, "vocab_size": 4096}, "model_name": "L12_H8_D320", "n_embd": 320, "n_head": 8, "n_layer": 12, "parameters": 17367040, "run_mode": "locked_stream", "seed": 1, "stage": 4, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_eval_loss": 3.9331329613924026, "train_loss_last": 4.117517948150635, "val_eval_loss": 4.1162367314100266}

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/summary.csv

@@ -0,0 +1,51 @@
+run_mode,condition,condition_kind,stage,token_limit,model_name,n_layer,n_head,n_embd,parameters,dropout_initial,dropout_final,dropout_schedule,n,mean_train_eval_loss,std_train_eval_loss,mean_val_eval_loss,std_val_eval_loss,mean_generalization_gap,std_generalization_gap
+locked_stream,low_decay_heuristic,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.895861119031906,0.0,5.179202765226364,0.0,1.283341646194458,0.0
+locked_stream,old_formula_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,4.232671082019806,0.0,5.232291169464588,0.0,0.9996200874447823,0.0
+locked_stream,probe_cal_blend_wikitext103,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.188,0.01,log_prefix_anchor,1,4.0190100222826,0.0,5.176444113254547,0.0,1.1574340909719467,0.0
+locked_stream,probe_cal_strict_wikitext103,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.075,0.0,log_prefix_anchor,1,3.7474771067500114,0.0,5.205722317099571,0.0,1.4582452103495598,0.0
+locked_stream,static_dropout_0,static,0,250000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.466982416808605,0.0,5.355821952223778,0.0,1.8888395354151726,0.0
+locked_stream,static_dropout_0.02,static,0,250000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.596805490553379,0.0,5.286626905202866,0.0,1.6898214146494865,0.0
+locked_stream,static_dropout_0.04,static,0,250000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.6664711385965347,0.0,5.233644910156727,0.0,1.567173771560192,0.0
+locked_stream,static_dropout_0.08,static,0,250000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.754931300878525,0.0,5.194120541214943,0.0,1.4391892403364182,0.0
+locked_stream,static_dropout_0.14,static,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.8958611339330673,0.0,5.179202765226364,0.0,1.2833416312932968,0.0
+locked_stream,static_dropout_0.2,static,0,250000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,4.038728557527065,0.0,5.176292642951012,0.0,1.1375640854239464,0.0
+locked_stream,low_decay_heuristic,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.6468456014990807,0.0,4.78017570823431,0.0,1.1333301067352295,0.0
+locked_stream,old_formula_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,3.887567311525345,0.0,4.7666677832603455,0.0,0.8791004717350006,0.0
+locked_stream,probe_cal_blend_wikitext103,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.188,0.01,log_prefix_anchor,1,3.651739940047264,0.0,4.755539648234844,0.0,1.10379970818758,0.0
+locked_stream,probe_cal_strict_wikitext103,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.075,0.0,log_prefix_anchor,1,3.320544056594372,0.0,4.92121696472168,0.0,1.600672908127308,0.0
+locked_stream,static_dropout_0,static,1,500000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.120699368417263,0.0,5.164503566920757,0.0,2.0438041985034943,0.0
+locked_stream,static_dropout_0.02,static,1,500000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.2595799192786217,0.0,5.033853515982628,0.0,1.7742735967040062,0.0
+locked_stream,static_dropout_0.04,static,1,500000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.3200313225388527,0.0,4.927406623959541,0.0,1.6073753014206886,0.0
+locked_stream,static_dropout_0.08,static,1,500000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.474463567137718,0.0,4.827699206769466,0.0,1.3532356396317482,0.0
+locked_stream,static_dropout_0.14,static,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.6468456238508224,0.0,4.780175752937794,0.0,1.1333301290869713,0.0
+locked_stream,static_dropout_0.2,static,1,500000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,3.7633390352129936,0.0,4.751599572598934,0.0,0.9882605373859406,0.0
+locked_stream,low_decay_heuristic,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.653651036322117,0.0,4.508680582046509,0.0,0.8550295457243919,0.0
+locked_stream,old_formula_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,3.826009936630726,0.0,4.5063412338495255,0.0,0.6803312972187996,0.0
+locked_stream,probe_cal_blend_wikitext103,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.188,0.01,log_prefix_anchor,1,3.638385131955147,0.0,4.490205697715282,0.0,0.8518205657601357,0.0
+locked_stream,probe_cal_strict_wikitext103,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.075,0.0,log_prefix_anchor,1,3.373075522482395,0.0,4.705984242260456,0.0,1.332908719778061,0.0
+locked_stream,static_dropout_0,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.334706276655197,0.0,4.793723180890083,0.0,1.4590169042348862,0.0
+locked_stream,static_dropout_0.02,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.4418885335326195,0.0,4.665390767157078,0.0,1.2235022336244583,0.0
+locked_stream,static_dropout_0.04,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.5105278119444847,0.0,4.600722283124924,0.0,1.090194471180439,0.0
+locked_stream,static_dropout_0.08,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.5962011739611626,0.0,4.531755141913891,0.0,0.9355539679527283,0.0
+locked_stream,static_dropout_0.14,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.711411625146866,0.0,4.507372200489044,0.0,0.7959605753421783,0.0
+locked_stream,static_dropout_0.2,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,3.8255888894200325,0.0,4.5107810869812965,0.0,0.685192197561264,0.0
+locked_stream,low_decay_heuristic,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.7482572942972183,0.0,4.287984453141689,0.0,0.539727158844471,0.0
+locked_stream,old_formula_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,3.8000189810991287,0.0,4.282629065215588,0.0,0.4826100841164589,0.0
+locked_stream,probe_cal_blend_wikitext103,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.188,0.01,log_prefix_anchor,1,3.7199412807822227,0.0,4.287429004907608,0.0,0.5674877241253853,0.0
+locked_stream,probe_cal_strict_wikitext103,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.075,0.0,log_prefix_anchor,1,3.6160357668995857,0.0,4.402302749454975,0.0,0.7862669825553894,0.0
+locked_stream,static_dropout_0,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.616692140698433,0.0,4.428123250603676,0.0,0.8114311099052429,0.0
+locked_stream,static_dropout_0.02,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.6569968461990356,0.0,4.387734830379486,0.0,0.7307379841804504,0.0
+locked_stream,static_dropout_0.04,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.674849957227707,0.0,4.338778018951416,0.0,0.6639280617237091,0.0
+locked_stream,static_dropout_0.08,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.7277798801660538,0.0,4.301640450954437,0.0,0.5738605707883835,0.0
+locked_stream,static_dropout_0.14,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.8262650296092033,0.0,4.2931585647165775,0.0,0.4668935351073742,0.0
+locked_stream,static_dropout_0.2,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,3.8944422900676727,0.0,4.308795563876629,0.0,0.41435327380895615,0.0
+locked_stream,low_decay_heuristic,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,1,3.7924038767814636,0.0,4.085419524461031,0.0,0.29301564767956734,0.0
+locked_stream,old_formula_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.301,0.02,log_prefix_anchor,1,3.8212324753403664,0.0,4.064255967736244,0.0,0.24302349239587784,0.0
+locked_stream,probe_cal_blend_wikitext103,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.188,0.01,log_prefix_anchor,1,3.7707679346203804,0.0,4.068912364542484,0.0,0.2981444299221039,0.0
+locked_stream,probe_cal_strict_wikitext103,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.075,0.0,log_prefix_anchor,1,3.7672162503004074,0.0,4.144088886678219,0.0,0.37687263637781143,0.0
+locked_stream,static_dropout_0,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,1,3.7706343308091164,0.0,4.159992642700672,0.0,0.3893583118915558,0.0
+locked_stream,static_dropout_0.02,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,1,3.80513883382082,0.0,4.137094311416149,0.0,0.3319554775953293,0.0
+locked_stream,static_dropout_0.04,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,1,3.8134910240769386,0.0,4.103101808577776,0.0,0.2896107845008373,0.0
+locked_stream,static_dropout_0.08,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,1,3.844335936009884,0.0,4.089278068393469,0.0,0.24494213238358498,0.0
+locked_stream,static_dropout_0.14,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,1,3.885074496269226,0.0,4.1082155816257,0.0,0.22314108535647392,0.0
+locked_stream,static_dropout_0.2,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,1,3.9331329613924026,0.0,4.1162367314100266,0.0,0.1831037700176239,0.0

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/summary.json

@@ -0,0 +1,1102 @@
+[
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.895861119031906,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.179202765226364,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.283341646194458,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.232671082019806,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.232291169464588,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9996200874447823,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.188,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 4.0190100222826,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.176444113254547,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1574340909719467,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.075,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7474771067500114,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.205722317099571,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.4582452103495598,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.466982416808605,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.355821952223778,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.8888395354151726,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.596805490553379,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.286626905202866,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.6898214146494865,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.6664711385965347,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.233644910156727,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.567173771560192,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.754931300878525,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.194120541214943,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.4391892403364182,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8958611339330673,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.179202765226364,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.2833416312932968,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 4.038728557527065,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.176292642951012,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1375640854239464,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.6468456014990807,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.78017570823431,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1333301067352295,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.887567311525345,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.7666677832603455,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8791004717350006,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.188,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.651739940047264,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.755539648234844,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.10379970818758,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.075,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.320544056594372,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.92121696472168,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.600672908127308,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.120699368417263,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.164503566920757,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 2.0438041985034943,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.2595799192786217,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 5.033853515982628,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.7742735967040062,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.3200313225388527,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.927406623959541,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.6073753014206886,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.474463567137718,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.827699206769466,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.3532356396317482,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.6468456238508224,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.780175752937794,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.1333301290869713,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.7633390352129936,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.751599572598934,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9882605373859406,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.653651036322117,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.508680582046509,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8550295457243919,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.826009936630726,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.5063412338495255,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6803312972187996,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.188,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.638385131955147,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.490205697715282,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8518205657601357,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.075,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.373075522482395,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.705984242260456,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.332908719778061,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.334706276655197,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.793723180890083,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.4590169042348862,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.4418885335326195,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.665390767157078,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.2235022336244583,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.5105278119444847,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.600722283124924,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 1.090194471180439,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.5962011739611626,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.531755141913891,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.9355539679527283,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.711411625146866,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.507372200489044,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7959605753421783,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8255888894200325,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.5107810869812965,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.685192197561264,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7482572942972183,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.287984453141689,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.539727158844471,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.8000189810991287,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.282629065215588,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4826100841164589,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.188,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7199412807822227,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.287429004907608,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5674877241253853,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.075,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.6160357668995857,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.402302749454975,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7862669825553894,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.616692140698433,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.428123250603676,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.8114311099052429,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.6569968461990356,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.387734830379486,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.7307379841804504,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.674849957227707,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.338778018951416,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.6639280617237091,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.7277798801660538,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.301640450954437,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.5738605707883835,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8262650296092033,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.2931585647165775,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.4668935351073742,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8944422900676727,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.308795563876629,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.41435327380895615,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "low_decay_heuristic",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7924038767814636,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.085419524461031,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.29301564767956734,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "old_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.301,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.8212324753403664,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.064255967736244,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.24302349239587784,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_blend_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.188,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7707679346203804,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.068912364542484,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2981444299221039,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "probe_cal_strict_wikitext103",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.075,
+    "dropout_final": 0.0,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 1,
+    "mean_train_eval_loss": 3.7672162503004074,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.144088886678219,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.37687263637781143,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.7706343308091164,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.159992642700672,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3893583118915558,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.80513883382082,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.137094311416149,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.3319554775953293,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.8134910240769386,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.103101808577776,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.2896107845008373,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.844335936009884,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.089278068393469,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.24494213238358498,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.885074496269226,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.1082155816257,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.22314108535647392,
+    "std_generalization_gap": 0.0
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 1,
+    "mean_train_eval_loss": 3.9331329613924026,
+    "std_train_eval_loss": 0.0,
+    "mean_val_eval_loss": 4.1162367314100266,
+    "std_val_eval_loss": 0.0,
+    "mean_generalization_gap": 0.1831037700176239,
+    "std_generalization_gap": 0.0
+  }
+]

runs/probe_calibrated_stream_wikitext103/locked_stream/20260529-110250/trace.jsonl

@@ -0,0 +1,100 @@
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.868658065795898}
+{"condition": "old_formula_l12", "dropout": 0.301, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.400547981262207}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.393429756164551}
+{"condition": "old_formula_l12", "dropout": 0.255, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.966970443725586}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.019368648529053}
+{"condition": "old_formula_l12", "dropout": 0.177, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.897003412246704}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.998159170150757}
+{"condition": "old_formula_l12", "dropout": 0.087, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.769676923751831}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.8899354934692383}
+{"condition": "old_formula_l12", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.9641218185424805}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.075, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.576968669891357}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.075, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.656510591506958}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.028999999999999998, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.027482509613037}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.028999999999999998, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.5648581981658936}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.00278377532959}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.1880264282226562}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.801042079925537}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.779501438140869}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.8576889038085938}
+{"condition": "probe_cal_strict_wikitext103", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.632549524307251}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.188, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.72838830947876}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.188, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.025996208190918}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.142, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.04885196685791}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.142, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.9967105388641357}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.8322267532348633}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.089, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.895059823989868}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.850735664367676}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.044, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.9355649948120117}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.8069090843200684}
+{"condition": "probe_cal_blend_wikitext103", "dropout": 0.01, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.897859811782837}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.796130180358887}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.8898677825927734}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.075655937194824}
+{"condition": "low_decay_heuristic", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.756572723388672}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.965989112854004}
+{"condition": "low_decay_heuristic", "dropout": 0.1, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.709557056427002}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.068553924560547}
+{"condition": "low_decay_heuristic", "dropout": 0.06, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.707911252975464}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.7908525466918945}
+{"condition": "low_decay_heuristic", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.6913557052612305}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.586461544036865}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.4980170726776123}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.7061855792999268}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.010756015777588}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.8464653491973877}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.2843174934387207}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.748642921447754}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.6820335388183594}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.587371349334717}
+{"condition": "static_dropout_0", "dropout": 0.0, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.5455589294433594}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.596406936645508}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.5398495197296143}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.487887382507324}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.1747989654541016}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.5900022983551025}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.477747917175293}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.7301249504089355}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.790534496307373}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.079158782958984}
+{"condition": "static_dropout_0.02", "dropout": 0.02, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.661900281906128}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.564645767211914}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.7955689430236816}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.6338186264038086}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.7024903297424316}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.983562469482422}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.3805878162384033}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 3.712494373321533}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.938136577606201}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.9665350914001465}
+{"condition": "static_dropout_0.04", "dropout": 0.04, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.9043874740600586}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.694372177124023}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.008996963500977}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 3.710146427154541}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.538743495941162}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 3.9808402061462402}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.636242389678955}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.011699676513672}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.6634957790374756}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.111110687255859}
+{"condition": "static_dropout_0.08", "dropout": 0.08, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.8889031410217285}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.7961297035217285}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 3.8898675441741943}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.075655460357666}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.75657320022583}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.029130935668945}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.773858070373535}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.1858930587768555}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.8487484455108643}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 3.9703288078308105}
+{"condition": "static_dropout_0.14", "dropout": 0.14, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 3.8854050636291504}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 500, "steps": 1000, "token_limit": 250000, "tokens_seen": 1024000, "train_batch_loss": 4.769725799560547}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 0, "step": 1000, "steps": 1000, "token_limit": 250000, "tokens_seen": 2048000, "train_batch_loss": 4.0768561363220215}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 500, "steps": 1000, "token_limit": 500000, "tokens_seen": 3072000, "train_batch_loss": 4.300280570983887}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 1, "step": 1000, "steps": 1000, "token_limit": 500000, "tokens_seen": 4096000, "train_batch_loss": 3.8637547492980957}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 500, "steps": 1000, "token_limit": 1000000, "tokens_seen": 5120000, "train_batch_loss": 4.15064001083374}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 2, "step": 1000, "steps": 1000, "token_limit": 1000000, "tokens_seen": 6144000, "train_batch_loss": 3.981992483139038}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 500, "steps": 1000, "token_limit": 2000000, "tokens_seen": 7168000, "train_batch_loss": 4.188844203948975}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 3, "step": 1000, "steps": 1000, "token_limit": 2000000, "tokens_seen": 8192000, "train_batch_loss": 3.994189500808716}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 500, "steps": 1000, "token_limit": 4000000, "tokens_seen": 9216000, "train_batch_loss": 4.077834129333496}
+{"condition": "static_dropout_0.2", "dropout": 0.2, "event": "train_step", "model_name": "L12_H8_D320", "run_mode": "locked_stream", "seed": 1, "stage": 4, "step": 1000, "steps": 1000, "token_limit": 4000000, "tokens_seen": 10240000, "train_batch_loss": 4.117517948150635}