Add WikiText-103 five-seed streaming validation

docs/plan.md

@@ -285,7 +285,7 @@ Use this order for every regime.
 | TinyStories static/coefficient regime | active | main coefficient evidence |
 | TinyStories streaming regime | 5-seed validation complete | current main streaming evidence; interaction decay beats best static in 5/5 paired final-loss comparisons |
 | OpenWebText10K streaming regime | 5-seed clean validation complete | OpenWebText10K interaction decay beats best static in 5/5 paired final-loss comparisons |
-| WikiText-103 streaming regime | pending | start only after TinyStories and OpenWebText10K streaming reports are reconciled |
+| WikiText-103 streaming regime | 5-seed validation complete | formula-derived L12 decay beats best static in 5/5 paired final-loss comparisons |
 
 ## Current Formula Status
 
@@ -334,6 +334,7 @@ structure transfers, while coefficients may be regime-specific.
 | TinyStories held-out model | supports pressure dependence on model size |
 | TinyStories streaming, 5 seeds | interaction has best mean final loss; interaction beats best static in 5/5 paired final-loss comparisons |
 | OpenWebText10K streaming, 5 seeds | interaction decay has best mean final loss; top decay schedules beat best static in 5/5 paired comparisons |
+| WikiText-103 streaming, 5 seeds | formula-derived L12 decay has best mean final loss; beats best static in 5/5 paired comparisons |
 | cross-regime raw coefficient transfer | weaker than within-regime fit; supports regime-specific coefficients rather than universal numeric coefficients |
 
 Latest TinyStories 5-seed streaming final-loss table:
@@ -401,6 +402,38 @@ The best static baseline in the clean OpenWebText10K run is static dropout
 `0.0473` and wins every paired seed comparison. This promotes OpenWebText10K
 from exploratory support to a second multi-seed streaming validation regime.
 
+Latest WikiText-103 5-seed streaming final-loss table:
+
+| Condition | Mean final 4M validation loss | Std |
+|---|---:|---:|
+| `wikitext103_formula_l12` decay | 4.0808 | 0.0195 |
+| `wikitext103_probe_blend` decay | 4.0961 | 0.0145 |
+| `wikitext103_low_decay` decay | 4.1020 | 0.0166 |
+| static `0.10` | 4.1105 | 0.0188 |
+| static `0.08` | 4.1116 | 0.0186 |
+| static `0.06` | 4.1197 | 0.0082 |
+| static `0.14` | 4.1221 | 0.0155 |
+| static `0.18` | 4.1304 | 0.0130 |
+| static `0.04` | 4.1331 | 0.0227 |
+| static `0.20` | 4.1394 | 0.0167 |
+| static `0.02` | 4.1459 | 0.0165 |
+| static `0.26` | 4.1784 | 0.0145 |
+| static `0.00` | 4.1835 | 0.0165 |
+| static `0.30` | 4.1946 | 0.0141 |
+
+Paired final-loss result:
+
+| Decay schedule | Paired wins vs best static |
+|---|---:|
+| `wikitext103_formula_l12` | 5/5 |
+| `wikitext103_probe_blend` | 4/5 |
+| `wikitext103_low_decay` | 4/5 |
+
+The best static baseline in the clean WikiText-103 run is static dropout
+`0.10` by mean final loss. The formula-derived L12 decay improves mean final
+validation loss by about `0.0297` and wins every paired seed comparison. This
+promotes WikiText-103 to a third multi-seed streaming validation regime.
+
 ## Completed Static Backtest Gate
 
 The first offline coefficient backtest is complete. It is retained as supporting
@@ -423,11 +456,15 @@ streaming multi-seed reports for each regime.
 
 ## Immediate Next Action
 
-Reconcile the TinyStories five-seed report and OpenWebText10K five-seed report
-into the paper outline. The seed-count gap is now closed. The next empirical
-weakness is external validity, so the preferred next experiment is a third
-held-out regime with minimal coefficient calibration followed by narrowed
-multi-seed streaming validation.
+Reconcile the TinyStories, OpenWebText10K, and WikiText-103 five-seed streaming
+reports into the paper outline. The strongest current claim is now supported in
+three regimes: formula-derived or regime-fitted decay schedules beat the best
+static dropout baseline in paired five-seed final-loss comparisons.
+
+The next empirical weakness is no longer a missing third text regime. The next
+useful strengthening step is to test robustness across a controlled architecture
+or token-budget change inside one established corpus regime, while preserving
+the same MPS-only, five-seed validation standard.
 
 ## Next Training After Current Gate
 
@@ -438,8 +475,10 @@ limiting issue, use a third held-out regime for the next validation step:
 ```text
 completed: TinyStories 5-seed streaming report
 completed: OpenWebText10K 5-seed clean streaming report
-next: third held-out regime with minimal calibration
-avoid: broad new sweep before cross-regime report reconciliation
+completed: WikiText-103 5-seed clean streaming report
+next: reconcile three-regime evidence into the paper, then choose one narrowed
+      robustness test
+avoid: broad new sweep before three-regime report reconciliation
 ```
 
 Evaluate with paired seed comparisons:
@@ -452,17 +491,20 @@ decay minus best-static delta per seed
 rank consistency across seeds
 ```
 
-Because OpenWebText10K decay wins across paired seeds, promote the cross-regime
-streaming claim to "supported in two regimes." Do not yet claim universal
-numeric coefficients. The next claim to test is whether the pressure-law
-structure and regime-specific fitting procedure reproduce the win in a third
-held-out regime.
+Because TinyStories, OpenWebText10K, and WikiText-103 decays win across paired
+seeds, promote the cross-regime streaming claim to "supported in three regimes."
+Do not yet claim universal numeric coefficients. The current defensible
+paper-level claim is that the pressure-law structure and regime-specific fitting
+procedure can produce dropout schedules that beat the best static dropout
+baseline across multiple text regimes.
 
 Latest streaming report:
 
 ```text
 docs/tinystories_streaming_report.md
 docs/openwebtext10k_streaming_report.md
+docs/wikitext103_streaming_report.md
 runs/streaming_tinystories_multiseed_validation_l12/combined_5seed_summary/
 runs/openwebtext10k_streaming_report/l16_updated_formula_clean_5seed/
+runs/wikitext103_streaming_report/l12_validation_5seed/
 ```

docs/wikitext103_streaming_report.md

@@ -0,0 +1,198 @@
+# WikiText-103 Streaming Validation
+
+Date: 2026-05-31
+
+This report combines 5 random seeds (1, 2, 3, 4, 5) from saved streaming runs.
+No additional training is performed by this script; it reads saved
+`metrics.jsonl` files.
+
+Regime: WikiText-103 cached-corpus streaming setup with L12_H8_D320, 17,367,040 parameters, five prefixes from 250k to 4M tokens, and 1,000 optimizer steps per stage. This is a clean five-seed run including three dropout decay schedules and broad static dropout baselines from 0.00 through 0.30.
+
+## Sources
+
+- `runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525/metrics.jsonl`
+
+## Condition Ranking By Final Loss
+
+| Condition | Kind | N | Mean trajectory val | Std trajectory val | Mean final val | Std final val | Mean final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---:|---:|---|
+| `wikitext103_formula_l12` | `anchor_decay` | 5 | 4.5711 | 0.0045 | 4.0808 | 0.0195 | 0.2817 | `0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02` |
+| `wikitext103_probe_blend` | `anchor_decay` | 5 | 4.5635 | 0.0046 | 4.0961 | 0.0145 | 0.3287 | `0.19 -> 0.14 -> 0.09 -> 0.04 -> 0.01` |
+| `wikitext103_low_decay` | `anchor_decay` | 5 | 4.5681 | 0.0073 | 4.1020 | 0.0166 | 0.3251 | `0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02` |
+| `static_dropout_0.1` | `static` | 5 | 4.5836 | 0.0062 | 4.1105 | 0.0188 | 0.2687 | `0.10 -> 0.10 -> 0.10 -> 0.10 -> 0.10` |
+| `static_dropout_0.08` | `static` | 5 | 4.5967 | 0.0073 | 4.1116 | 0.0186 | 0.2848 | `0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08` |
+| `static_dropout_0.06` | `static` | 5 | 4.6186 | 0.0048 | 4.1197 | 0.0082 | 0.3131 | `0.06 -> 0.06 -> 0.06 -> 0.06 -> 0.06` |
+| `static_dropout_0.14` | `static` | 5 | 4.5735 | 0.0077 | 4.1221 | 0.0155 | 0.2548 | `0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14` |
+| `static_dropout_0.18` | `static` | 5 | 4.5756 | 0.0041 | 4.1304 | 0.0130 | 0.2289 | `0.18 -> 0.18 -> 0.18 -> 0.18 -> 0.18` |
+| `static_dropout_0.04` | `static` | 5 | 4.6501 | 0.0077 | 4.1331 | 0.0227 | 0.3353 | `0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04` |
+| `static_dropout_0.2` | `static` | 5 | 4.5794 | 0.0050 | 4.1394 | 0.0167 | 0.2239 | `0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20` |
+| `static_dropout_0.02` | `static` | 5 | 4.6954 | 0.0086 | 4.1459 | 0.0165 | 0.3700 | `0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02` |
+| `static_dropout_0.26` | `static` | 5 | 4.6063 | 0.0051 | 4.1784 | 0.0145 | 0.2008 | `0.26 -> 0.26 -> 0.26 -> 0.26 -> 0.26` |
+| `static_dropout_0` | `static` | 5 | 4.7762 | 0.0109 | 4.1835 | 0.0165 | 0.4085 | `0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00` |
+| `static_dropout_0.3` | `static` | 5 | 4.6253 | 0.0034 | 4.1946 | 0.0141 | 0.1819 | `0.30 -> 0.30 -> 0.30 -> 0.30 -> 0.30` |
+
+## Paired Final-Loss Deltas
+
+Negative `delta_vs_best_static` means the condition beat the best static
+baseline for that seed.
+
+| Seed | Condition | Final val | Best static | Best static final val | Delta vs best static |
+|---:|---|---:|---|---:|---:|
+| 1 | `wikitext103_formula_l12` | 4.0623 | `static_dropout_0.1` | 4.0807 | -0.0184 |
+| 1 | `wikitext103_probe_blend` | 4.0738 | `static_dropout_0.1` | 4.0807 | -0.0069 |
+| 1 | `wikitext103_low_decay` | 4.0854 | `static_dropout_0.1` | 4.0807 | +0.0047 |
+| 1 | `static_dropout_0.1` | 4.0807 | `static_dropout_0.1` | 4.0807 | +0.0000 |
+| 1 | `static_dropout_0.08` | 4.0893 | `static_dropout_0.1` | 4.0807 | +0.0086 |
+| 1 | `static_dropout_0.06` | 4.1112 | `static_dropout_0.1` | 4.0807 | +0.0305 |
+| 1 | `static_dropout_0.14` | 4.1082 | `static_dropout_0.1` | 4.0807 | +0.0275 |
+| 1 | `static_dropout_0.18` | 4.1108 | `static_dropout_0.1` | 4.0807 | +0.0301 |
+| 1 | `static_dropout_0.2` | 4.1162 | `static_dropout_0.1` | 4.0807 | +0.0355 |
+| 1 | `static_dropout_0.04` | 4.1031 | `static_dropout_0.1` | 4.0807 | +0.0224 |
+| 1 | `static_dropout_0.02` | 4.1371 | `static_dropout_0.1` | 4.0807 | +0.0564 |
+| 1 | `static_dropout_0` | 4.1600 | `static_dropout_0.1` | 4.0807 | +0.0793 |
+| 1 | `static_dropout_0.26` | 4.1557 | `static_dropout_0.1` | 4.0807 | +0.0750 |
+| 1 | `static_dropout_0.3` | 4.1802 | `static_dropout_0.1` | 4.0807 | +0.0994 |
+| 2 | `wikitext103_formula_l12` | 4.1123 | `static_dropout_0.06` | 4.1304 | -0.0181 |
+| 2 | `wikitext103_probe_blend` | 4.1113 | `static_dropout_0.06` | 4.1304 | -0.0191 |
+| 2 | `wikitext103_low_decay` | 4.1291 | `static_dropout_0.06` | 4.1304 | -0.0013 |
+| 2 | `static_dropout_0.1` | 4.1320 | `static_dropout_0.06` | 4.1304 | +0.0016 |
+| 2 | `static_dropout_0.08` | 4.1374 | `static_dropout_0.06` | 4.1304 | +0.0071 |
+| 2 | `static_dropout_0.06` | 4.1304 | `static_dropout_0.06` | 4.1304 | +0.0000 |
+| 2 | `static_dropout_0.14` | 4.1476 | `static_dropout_0.06` | 4.1304 | +0.0172 |
+| 2 | `static_dropout_0.18` | 4.1471 | `static_dropout_0.06` | 4.1304 | +0.0167 |
+| 2 | `static_dropout_0.2` | 4.1633 | `static_dropout_0.06` | 4.1304 | +0.0329 |
+| 2 | `static_dropout_0.04` | 4.1648 | `static_dropout_0.06` | 4.1304 | +0.0344 |
+| 2 | `static_dropout_0.02` | 4.1746 | `static_dropout_0.06` | 4.1304 | +0.0442 |
+| 2 | `static_dropout_0` | 4.2030 | `static_dropout_0.06` | 4.1304 | +0.0726 |
+| 2 | `static_dropout_0.26` | 4.1961 | `static_dropout_0.06` | 4.1304 | +0.0658 |
+| 2 | `static_dropout_0.3` | 4.2155 | `static_dropout_0.06` | 4.1304 | +0.0852 |
+| 3 | `wikitext103_formula_l12` | 4.0763 | `static_dropout_0.08` | 4.1036 | -0.0272 |
+| 3 | `wikitext103_probe_blend` | 4.0934 | `static_dropout_0.08` | 4.1036 | -0.0102 |
+| 3 | `wikitext103_low_decay` | 4.1006 | `static_dropout_0.08` | 4.1036 | -0.0030 |
+| 3 | `static_dropout_0.1` | 4.1115 | `static_dropout_0.08` | 4.1036 | +0.0079 |
+| 3 | `static_dropout_0.08` | 4.1036 | `static_dropout_0.08` | 4.1036 | +0.0000 |
+| 3 | `static_dropout_0.06` | 4.1127 | `static_dropout_0.08` | 4.1036 | +0.0092 |
+| 3 | `static_dropout_0.14` | 4.1240 | `static_dropout_0.08` | 4.1036 | +0.0204 |
+| 3 | `static_dropout_0.18` | 4.1285 | `static_dropout_0.08` | 4.1036 | +0.0250 |
+| 3 | `static_dropout_0.2` | 4.1367 | `static_dropout_0.08` | 4.1036 | +0.0332 |
+| 3 | `static_dropout_0.04` | 4.1246 | `static_dropout_0.08` | 4.1036 | +0.0211 |
+| 3 | `static_dropout_0.02` | 4.1443 | `static_dropout_0.08` | 4.1036 | +0.0408 |
+| 3 | `static_dropout_0` | 4.1758 | `static_dropout_0.08` | 4.1036 | +0.0722 |
+| 3 | `static_dropout_0.26` | 4.1796 | `static_dropout_0.08` | 4.1036 | +0.0761 |
+| 3 | `static_dropout_0.3` | 4.1926 | `static_dropout_0.08` | 4.1036 | +0.0890 |
+| 4 | `wikitext103_formula_l12` | 4.0845 | `static_dropout_0.1` | 4.1096 | -0.0251 |
+| 4 | `wikitext103_probe_blend` | 4.0954 | `static_dropout_0.1` | 4.1096 | -0.0141 |
+| 4 | `wikitext103_low_decay` | 4.0928 | `static_dropout_0.1` | 4.1096 | -0.0167 |
+| 4 | `static_dropout_0.1` | 4.1096 | `static_dropout_0.1` | 4.1096 | +0.0000 |
+| 4 | `static_dropout_0.08` | 4.1223 | `static_dropout_0.1` | 4.1096 | +0.0127 |
+| 4 | `static_dropout_0.06` | 4.1188 | `static_dropout_0.1` | 4.1096 | +0.0093 |
+| 4 | `static_dropout_0.14` | 4.1117 | `static_dropout_0.1` | 4.1096 | +0.0021 |
+| 4 | `static_dropout_0.18` | 4.1330 | `static_dropout_0.1` | 4.1096 | +0.0234 |
+| 4 | `static_dropout_0.2` | 4.1388 | `static_dropout_0.1` | 4.1096 | +0.0292 |
+| 4 | `static_dropout_0.04` | 4.1312 | `static_dropout_0.1` | 4.1096 | +0.0217 |
+| 4 | `static_dropout_0.02` | 4.1387 | `static_dropout_0.1` | 4.1096 | +0.0291 |
+| 4 | `static_dropout_0` | 4.1853 | `static_dropout_0.1` | 4.1096 | +0.0757 |
+| 4 | `static_dropout_0.26` | 4.1782 | `static_dropout_0.1` | 4.1096 | +0.0686 |
+| 4 | `static_dropout_0.3` | 4.2007 | `static_dropout_0.1` | 4.1096 | +0.0912 |
+| 5 | `wikitext103_formula_l12` | 4.0686 | `static_dropout_0.08` | 4.1056 | -0.0370 |
+| 5 | `wikitext103_probe_blend` | 4.1066 | `static_dropout_0.08` | 4.1056 | +0.0009 |
+| 5 | `wikitext103_low_decay` | 4.1021 | `static_dropout_0.08` | 4.1056 | -0.0035 |
+| 5 | `static_dropout_0.1` | 4.1186 | `static_dropout_0.08` | 4.1056 | +0.0129 |
+| 5 | `static_dropout_0.08` | 4.1056 | `static_dropout_0.08` | 4.1056 | +0.0000 |
+| 5 | `static_dropout_0.06` | 4.1253 | `static_dropout_0.08` | 4.1056 | +0.0197 |
+| 5 | `static_dropout_0.14` | 4.1192 | `static_dropout_0.08` | 4.1056 | +0.0135 |
+| 5 | `static_dropout_0.18` | 4.1325 | `static_dropout_0.08` | 4.1056 | +0.0269 |
+| 5 | `static_dropout_0.2` | 4.1418 | `static_dropout_0.08` | 4.1056 | +0.0362 |
+| 5 | `static_dropout_0.04` | 4.1419 | `static_dropout_0.08` | 4.1056 | +0.0363 |
+| 5 | `static_dropout_0.02` | 4.1346 | `static_dropout_0.08` | 4.1056 | +0.0290 |
+| 5 | `static_dropout_0` | 4.1934 | `static_dropout_0.08` | 4.1056 | +0.0878 |
+| 5 | `static_dropout_0.26` | 4.1821 | `static_dropout_0.08` | 4.1056 | +0.0765 |
+| 5 | `static_dropout_0.3` | 4.1841 | `static_dropout_0.08` | 4.1056 | +0.0785 |
+
+## Stage Trajectory
+
+| Stage | Prefix tokens | Condition | Dropout | N | Mean val | Std val | Mean train | Mean gap |
+|---:|---:|---|---:|---:|---:|---:|---:|---:|
+| 0 | 250,000 | `static_dropout_0.18` | 0.180 | 5 | 5.1616 | 0.0150 | 3.9964 | 1.1652 |
+| 0 | 250,000 | `wikitext103_low_decay` | 0.140 | 5 | 5.1635 | 0.0220 | 3.9051 | 1.2585 |
+| 0 | 250,000 | `static_dropout_0.14` | 0.140 | 5 | 5.1635 | 0.0220 | 3.9051 | 1.2585 |
+| 0 | 250,000 | `wikitext103_probe_blend` | 0.190 | 5 | 5.1659 | 0.0171 | 4.0201 | 1.1458 |
+| 0 | 250,000 | `static_dropout_0.1` | 0.100 | 5 | 5.1699 | 0.0237 | 3.8219 | 1.3480 |
+| 0 | 250,000 | `static_dropout_0.2` | 0.200 | 5 | 5.1701 | 0.0141 | 4.0363 | 1.1338 |
+| 0 | 250,000 | `static_dropout_0.08` | 0.080 | 5 | 5.1894 | 0.0161 | 3.7619 | 1.4274 |
+| 0 | 250,000 | `static_dropout_0.26` | 0.260 | 5 | 5.1940 | 0.0161 | 4.1496 | 1.0444 |
+| 0 | 250,000 | `wikitext103_formula_l12` | 0.300 | 5 | 5.2148 | 0.0181 | 4.2131 | 1.0017 |
+| 0 | 250,000 | `static_dropout_0.3` | 0.300 | 5 | 5.2148 | 0.0181 | 4.2131 | 1.0017 |
+| 0 | 250,000 | `static_dropout_0.06` | 0.060 | 5 | 5.2154 | 0.0173 | 3.7128 | 1.5026 |
+| 0 | 250,000 | `static_dropout_0.04` | 0.040 | 5 | 5.2378 | 0.0186 | 3.6441 | 1.5938 |
+| 0 | 250,000 | `static_dropout_0.02` | 0.020 | 5 | 5.2750 | 0.0255 | 3.5725 | 1.7025 |
+| 0 | 250,000 | `static_dropout_0` | 0.000 | 5 | 5.3403 | 0.0270 | 3.5230 | 1.8172 |
+| 1 | 500,000 | `wikitext103_probe_blend` | 0.140 | 5 | 4.7872 | 0.0269 | 3.6846 | 1.1027 |
+| 1 | 500,000 | `static_dropout_0.2` | 0.200 | 5 | 4.7873 | 0.0236 | 3.7914 | 0.9959 |
+| 1 | 500,000 | `static_dropout_0.18` | 0.180 | 5 | 4.7946 | 0.0206 | 3.7572 | 1.0375 |
+| 1 | 500,000 | `static_dropout_0.14` | 0.140 | 5 | 4.8001 | 0.0198 | 3.6650 | 1.1351 |
+| 1 | 500,000 | `wikitext103_low_decay` | 0.140 | 5 | 4.8001 | 0.0198 | 3.6650 | 1.1351 |
+| 1 | 500,000 | `wikitext103_formula_l12` | 0.260 | 5 | 4.8053 | 0.0278 | 3.9182 | 0.8871 |
+| 1 | 500,000 | `static_dropout_0.26` | 0.260 | 5 | 4.8081 | 0.0216 | 3.9053 | 0.9028 |
+| 1 | 500,000 | `static_dropout_0.3` | 0.300 | 5 | 4.8242 | 0.0296 | 3.9765 | 0.8476 |
+| 1 | 500,000 | `static_dropout_0.1` | 0.100 | 5 | 4.8332 | 0.0258 | 3.5637 | 1.2695 |
+| 1 | 500,000 | `static_dropout_0.08` | 0.080 | 5 | 4.8576 | 0.0239 | 3.5036 | 1.3540 |
+| 1 | 500,000 | `static_dropout_0.06` | 0.060 | 5 | 4.8947 | 0.0213 | 3.4394 | 1.4552 |
+| 1 | 500,000 | `static_dropout_0.04` | 0.040 | 5 | 4.9573 | 0.0250 | 3.3515 | 1.6058 |
+| 1 | 500,000 | `static_dropout_0.02` | 0.020 | 5 | 5.0451 | 0.0169 | 3.2612 | 1.7839 |
+| 1 | 500,000 | `static_dropout_0` | 0.000 | 5 | 5.1741 | 0.0252 | 3.1506 | 2.0235 |
+| 2 | 1,000,000 | `wikitext103_formula_l12` | 0.180 | 5 | 4.4938 | 0.0147 | 3.8283 | 0.6655 |
+| 2 | 1,000,000 | `wikitext103_probe_blend` | 0.090 | 5 | 4.4940 | 0.0159 | 3.6495 | 0.8445 |
+| 2 | 1,000,000 | `static_dropout_0.14` | 0.140 | 5 | 4.5001 | 0.0148 | 3.7163 | 0.7838 |
+| 2 | 1,000,000 | `wikitext103_low_decay` | 0.100 | 5 | 4.5013 | 0.0158 | 3.6607 | 0.8406 |
+| 2 | 1,000,000 | `static_dropout_0.18` | 0.180 | 5 | 4.5023 | 0.0185 | 3.7866 | 0.7157 |
+| 2 | 1,000,000 | `static_dropout_0.2` | 0.200 | 5 | 4.5060 | 0.0204 | 3.8148 | 0.6913 |
+| 2 | 1,000,000 | `static_dropout_0.1` | 0.100 | 5 | 4.5186 | 0.0135 | 3.6524 | 0.8662 |
+| 2 | 1,000,000 | `static_dropout_0.26` | 0.260 | 5 | 4.5262 | 0.0101 | 3.9071 | 0.6191 |
+| 2 | 1,000,000 | `static_dropout_0.08` | 0.080 | 5 | 4.5326 | 0.0117 | 3.6000 | 0.9326 |
+| 2 | 1,000,000 | `static_dropout_0.3` | 0.300 | 5 | 4.5462 | 0.0127 | 3.9708 | 0.5754 |
+| 2 | 1,000,000 | `static_dropout_0.06` | 0.060 | 5 | 4.5574 | 0.0126 | 3.5554 | 1.0020 |
+| 2 | 1,000,000 | `static_dropout_0.04` | 0.040 | 5 | 4.5959 | 0.0146 | 3.5030 | 1.0929 |
+| 2 | 1,000,000 | `static_dropout_0.02` | 0.020 | 5 | 4.6558 | 0.0159 | 3.4324 | 1.2234 |
+| 2 | 1,000,000 | `static_dropout_0` | 0.000 | 5 | 4.7661 | 0.0324 | 3.3658 | 1.4003 |
+| 3 | 2,000,000 | `wikitext103_formula_l12` | 0.090 | 5 | 4.2607 | 0.0181 | 3.8089 | 0.4518 |
+| 3 | 2,000,000 | `wikitext103_low_decay` | 0.060 | 5 | 4.2736 | 0.0228 | 3.7474 | 0.5261 |
+| 3 | 2,000,000 | `wikitext103_probe_blend` | 0.040 | 5 | 4.2743 | 0.0174 | 3.7200 | 0.5543 |
+| 3 | 2,000,000 | `static_dropout_0.14` | 0.140 | 5 | 4.2816 | 0.0214 | 3.8287 | 0.4529 |
+| 3 | 2,000,000 | `static_dropout_0.1` | 0.100 | 5 | 4.2857 | 0.0190 | 3.7809 | 0.5048 |
+| 3 | 2,000,000 | `static_dropout_0.18` | 0.180 | 5 | 4.2889 | 0.0195 | 3.8768 | 0.4121 |
+| 3 | 2,000,000 | `static_dropout_0.08` | 0.080 | 5 | 4.2925 | 0.0166 | 3.7519 | 0.5406 |
+| 3 | 2,000,000 | `static_dropout_0.2` | 0.200 | 5 | 4.2942 | 0.0160 | 3.8982 | 0.3960 |
+| 3 | 2,000,000 | `static_dropout_0.06` | 0.060 | 5 | 4.3059 | 0.0182 | 3.7318 | 0.5741 |
+| 3 | 2,000,000 | `static_dropout_0.26` | 0.260 | 5 | 4.3248 | 0.0179 | 3.9655 | 0.3593 |
+| 3 | 2,000,000 | `static_dropout_0.04` | 0.040 | 5 | 4.3262 | 0.0165 | 3.7038 | 0.6225 |
+| 3 | 2,000,000 | `static_dropout_0.3` | 0.300 | 5 | 4.3467 | 0.0159 | 4.0097 | 0.3370 |
+| 3 | 2,000,000 | `static_dropout_0.02` | 0.020 | 5 | 4.3551 | 0.0265 | 3.6673 | 0.6878 |
+| 3 | 2,000,000 | `static_dropout_0` | 0.000 | 5 | 4.4174 | 0.0188 | 3.6409 | 0.7765 |
+| 4 | 4,000,000 | `wikitext103_formula_l12` | 0.020 | 5 | 4.0808 | 0.0195 | 3.7991 | 0.2817 |
+| 4 | 4,000,000 | `wikitext103_probe_blend` | 0.010 | 5 | 4.0961 | 0.0145 | 3.7674 | 0.3287 |
+| 4 | 4,000,000 | `wikitext103_low_decay` | 0.020 | 5 | 4.1020 | 0.0166 | 3.7769 | 0.3251 |
+| 4 | 4,000,000 | `static_dropout_0.1` | 0.100 | 5 | 4.1105 | 0.0188 | 3.8417 | 0.2687 |
+| 4 | 4,000,000 | `static_dropout_0.08` | 0.080 | 5 | 4.1116 | 0.0186 | 3.8268 | 0.2848 |
+| 4 | 4,000,000 | `static_dropout_0.06` | 0.060 | 5 | 4.1197 | 0.0082 | 3.8066 | 0.3131 |
+| 4 | 4,000,000 | `static_dropout_0.14` | 0.140 | 5 | 4.1221 | 0.0155 | 3.8674 | 0.2548 |
+| 4 | 4,000,000 | `static_dropout_0.18` | 0.180 | 5 | 4.1304 | 0.0130 | 3.9015 | 0.2289 |
+| 4 | 4,000,000 | `static_dropout_0.04` | 0.040 | 5 | 4.1331 | 0.0227 | 3.7978 | 0.3353 |
+| 4 | 4,000,000 | `static_dropout_0.2` | 0.200 | 5 | 4.1394 | 0.0167 | 3.9155 | 0.2239 |
+| 4 | 4,000,000 | `static_dropout_0.02` | 0.020 | 5 | 4.1459 | 0.0165 | 3.7759 | 0.3700 |
+| 4 | 4,000,000 | `static_dropout_0.26` | 0.260 | 5 | 4.1784 | 0.0145 | 3.9775 | 0.2008 |
+| 4 | 4,000,000 | `static_dropout_0` | 0.000 | 5 | 4.1835 | 0.0165 | 3.7750 | 0.4085 |
+| 4 | 4,000,000 | `static_dropout_0.3` | 0.300 | 5 | 4.1946 | 0.0141 | 4.0127 | 0.1819 |
+
+## Interpretation
+
+- `wikitext103_formula_l12` has the best 5-seed mean final validation loss: 4.0808 +/- 0.0195.
+- The second-best final condition is `wikitext103_probe_blend` at 4.0961 +/- 0.0145.
+- The best static baseline by mean final loss is `static_dropout_0.1` at 4.1105 +/- 0.0188.
+- `wikitext103_formula_l12` beats the per-seed best static baseline in 5/5 seeds; worst paired delta is -0.0181.
+- `wikitext103_probe_blend` beats the per-seed best static baseline in 4/5 seeds; worst paired delta is +0.0009.
+- `wikitext103_low_decay` beats the per-seed best static baseline in 4/5 seeds; worst paired delta is +0.0047.
+- The best first-stage condition is `static_dropout_0.18` at prefix 250,000 with mean validation loss 5.1616; compare this with the final ranking before claiming a schedule is uniformly better.
+- This is a saved-run streaming validation artifact. Treat it as strong
+  evidence only when the tested conditions, seeds, static baselines, and
+  stream protocol match the claim being made.

runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525/RESULT_SUMMARY.md

@@ -0,0 +1,122 @@
+# Locked Streaming Dropout Summary
+
+Run directory: `runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525`
+
+Model: `L12_H8_D320` causal Transformer, 17,367,040 parameters, 12 layers, 8 heads, 320 embedding dim.
+Training per stage: 1,000 steps. Sampled tokens are cumulative in each stage row. Seeds present: 1, 2, 3, 4, 5.
+
+## Condition Ranking
+
+| Condition | Kind | Final dropout | Mean trajectory val loss | Final val loss | Final gap | Dropout path |
+|---|---|---:|---:|---:|---:|---|
+| `wikitext103_probe_blend` | anchor_decay | 0.01 | 4.5635 | 4.0961 | 0.3287 | 0.19 -> 0.14 -> 0.09 -> 0.04 -> 0.01 |
+| `wikitext103_low_decay` | anchor_decay | 0.02 | 4.5681 | 4.1020 | 0.3251 | 0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02 |
+| `wikitext103_formula_l12` | anchor_decay | 0.02 | 4.5711 | 4.0808 | 0.2817 | 0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02 |
+| `static_dropout_0.14` | static | 0.14 | 4.5735 | 4.1221 | 0.2548 | 0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14 |
+| `static_dropout_0.18` | static | 0.18 | 4.5756 | 4.1304 | 0.2289 | 0.18 -> 0.18 -> 0.18 -> 0.18 -> 0.18 |
+| `static_dropout_0.2` | static | 0.20 | 4.5794 | 4.1394 | 0.2239 | 0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20 |
+| `static_dropout_0.1` | static | 0.10 | 4.5836 | 4.1105 | 0.2687 | 0.10 -> 0.10 -> 0.10 -> 0.10 -> 0.10 |
+| `static_dropout_0.08` | static | 0.08 | 4.5967 | 4.1116 | 0.2848 | 0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08 |
+| `static_dropout_0.26` | static | 0.26 | 4.6063 | 4.1784 | 0.2008 | 0.26 -> 0.26 -> 0.26 -> 0.26 -> 0.26 |
+| `static_dropout_0.06` | static | 0.06 | 4.6186 | 4.1197 | 0.3131 | 0.06 -> 0.06 -> 0.06 -> 0.06 -> 0.06 |
+| `static_dropout_0.3` | static | 0.30 | 4.6253 | 4.1946 | 0.1819 | 0.30 -> 0.30 -> 0.30 -> 0.30 -> 0.30 |
+| `static_dropout_0.04` | static | 0.04 | 4.6501 | 4.1331 | 0.3353 | 0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04 |
+| `static_dropout_0.02` | static | 0.02 | 4.6954 | 4.1459 | 0.3700 | 0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02 |
+| `static_dropout_0` | static | 0.00 | 4.7762 | 4.1835 | 0.4085 | 0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00 |
+
+## Stage Trajectory
+
+### Stage 0: 250,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `static_dropout_0.18` | 0.18 | 5.1616 | 3.9964 | 1.1652 | 5 |
+| `wikitext103_low_decay` | 0.14 | 5.1635 | 3.9051 | 1.2585 | 5 |
+| `static_dropout_0.14` | 0.14 | 5.1635 | 3.9051 | 1.2585 | 5 |
+| `wikitext103_probe_blend` | 0.19 | 5.1659 | 4.0201 | 1.1458 | 5 |
+| `static_dropout_0.1` | 0.10 | 5.1699 | 3.8219 | 1.3480 | 5 |
+| `static_dropout_0.2` | 0.20 | 5.1701 | 4.0363 | 1.1338 | 5 |
+| `static_dropout_0.08` | 0.08 | 5.1894 | 3.7619 | 1.4274 | 5 |
+| `static_dropout_0.26` | 0.26 | 5.1940 | 4.1496 | 1.0444 | 5 |
+| `wikitext103_formula_l12` | 0.30 | 5.2148 | 4.2131 | 1.0017 | 5 |
+| `static_dropout_0.3` | 0.30 | 5.2148 | 4.2131 | 1.0017 | 5 |
+| `static_dropout_0.06` | 0.06 | 5.2154 | 3.7128 | 1.5026 | 5 |
+| `static_dropout_0.04` | 0.04 | 5.2378 | 3.6441 | 1.5938 | 5 |
+| `static_dropout_0.02` | 0.02 | 5.2750 | 3.5725 | 1.7025 | 5 |
+| `static_dropout_0` | 0.00 | 5.3403 | 3.5230 | 1.8172 | 5 |
+
+### Stage 1: 500,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `wikitext103_probe_blend` | 0.14 | 4.7872 | 3.6846 | 1.1027 | 5 |
+| `static_dropout_0.2` | 0.20 | 4.7873 | 3.7914 | 0.9959 | 5 |
+| `static_dropout_0.18` | 0.18 | 4.7946 | 3.7572 | 1.0375 | 5 |
+| `static_dropout_0.14` | 0.14 | 4.8001 | 3.6650 | 1.1351 | 5 |
+| `wikitext103_low_decay` | 0.14 | 4.8001 | 3.6650 | 1.1351 | 5 |
+| `wikitext103_formula_l12` | 0.26 | 4.8053 | 3.9182 | 0.8871 | 5 |
+| `static_dropout_0.26` | 0.26 | 4.8081 | 3.9053 | 0.9028 | 5 |
+| `static_dropout_0.3` | 0.30 | 4.8242 | 3.9765 | 0.8476 | 5 |
+| `static_dropout_0.1` | 0.10 | 4.8332 | 3.5637 | 1.2695 | 5 |
+| `static_dropout_0.08` | 0.08 | 4.8576 | 3.5036 | 1.3540 | 5 |
+| `static_dropout_0.06` | 0.06 | 4.8947 | 3.4394 | 1.4552 | 5 |
+| `static_dropout_0.04` | 0.04 | 4.9573 | 3.3515 | 1.6058 | 5 |
+| `static_dropout_0.02` | 0.02 | 5.0451 | 3.2612 | 1.7839 | 5 |
+| `static_dropout_0` | 0.00 | 5.1741 | 3.1506 | 2.0235 | 5 |
+
+### Stage 2: 1,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `wikitext103_formula_l12` | 0.18 | 4.4938 | 3.8283 | 0.6655 | 5 |
+| `wikitext103_probe_blend` | 0.09 | 4.4940 | 3.6495 | 0.8445 | 5 |
+| `static_dropout_0.14` | 0.14 | 4.5001 | 3.7163 | 0.7838 | 5 |
+| `wikitext103_low_decay` | 0.10 | 4.5013 | 3.6607 | 0.8406 | 5 |
+| `static_dropout_0.18` | 0.18 | 4.5023 | 3.7866 | 0.7157 | 5 |
+| `static_dropout_0.2` | 0.20 | 4.5060 | 3.8148 | 0.6913 | 5 |
+| `static_dropout_0.1` | 0.10 | 4.5186 | 3.6524 | 0.8662 | 5 |
+| `static_dropout_0.26` | 0.26 | 4.5262 | 3.9071 | 0.6191 | 5 |
+| `static_dropout_0.08` | 0.08 | 4.5326 | 3.6000 | 0.9326 | 5 |
+| `static_dropout_0.3` | 0.30 | 4.5462 | 3.9708 | 0.5754 | 5 |
+| `static_dropout_0.06` | 0.06 | 4.5574 | 3.5554 | 1.0020 | 5 |
+| `static_dropout_0.04` | 0.04 | 4.5959 | 3.5030 | 1.0929 | 5 |
+| `static_dropout_0.02` | 0.02 | 4.6558 | 3.4324 | 1.2234 | 5 |
+| `static_dropout_0` | 0.00 | 4.7661 | 3.3658 | 1.4003 | 5 |
+
+### Stage 3: 2,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `wikitext103_formula_l12` | 0.09 | 4.2607 | 3.8089 | 0.4518 | 5 |
+| `wikitext103_low_decay` | 0.06 | 4.2736 | 3.7474 | 0.5261 | 5 |
+| `wikitext103_probe_blend` | 0.04 | 4.2743 | 3.7200 | 0.5543 | 5 |
+| `static_dropout_0.14` | 0.14 | 4.2816 | 3.8287 | 0.4529 | 5 |
+| `static_dropout_0.1` | 0.10 | 4.2857 | 3.7809 | 0.5048 | 5 |
+| `static_dropout_0.18` | 0.18 | 4.2889 | 3.8768 | 0.4121 | 5 |
+| `static_dropout_0.08` | 0.08 | 4.2925 | 3.7519 | 0.5406 | 5 |
+| `static_dropout_0.2` | 0.20 | 4.2942 | 3.8982 | 0.3960 | 5 |
+| `static_dropout_0.06` | 0.06 | 4.3059 | 3.7318 | 0.5741 | 5 |
+| `static_dropout_0.26` | 0.26 | 4.3248 | 3.9655 | 0.3593 | 5 |
+| `static_dropout_0.04` | 0.04 | 4.3262 | 3.7038 | 0.6225 | 5 |
+| `static_dropout_0.3` | 0.30 | 4.3467 | 4.0097 | 0.3370 | 5 |
+| `static_dropout_0.02` | 0.02 | 4.3551 | 3.6673 | 0.6878 | 5 |
+| `static_dropout_0` | 0.00 | 4.4174 | 3.6409 | 0.7765 | 5 |
+
+### Stage 4: 4,000,000 Prefix Tokens
+
+| Condition | Dropout | Mean val loss | Mean train loss | Mean gap | N |
+|---|---:|---:|---:|---:|---:|
+| `wikitext103_formula_l12` | 0.02 | 4.0808 | 3.7991 | 0.2817 | 5 |
+| `wikitext103_probe_blend` | 0.01 | 4.0961 | 3.7674 | 0.3287 | 5 |
+| `wikitext103_low_decay` | 0.02 | 4.1020 | 3.7769 | 0.3251 | 5 |
+| `static_dropout_0.1` | 0.10 | 4.1105 | 3.8417 | 0.2687 | 5 |
+| `static_dropout_0.08` | 0.08 | 4.1116 | 3.8268 | 0.2848 | 5 |
+| `static_dropout_0.06` | 0.06 | 4.1197 | 3.8066 | 0.3131 | 5 |
+| `static_dropout_0.14` | 0.14 | 4.1221 | 3.8674 | 0.2548 | 5 |
+| `static_dropout_0.18` | 0.18 | 4.1304 | 3.9015 | 0.2289 | 5 |
+| `static_dropout_0.04` | 0.04 | 4.1331 | 3.7978 | 0.3353 | 5 |
+| `static_dropout_0.2` | 0.20 | 4.1394 | 3.9155 | 0.2239 | 5 |
+| `static_dropout_0.02` | 0.02 | 4.1459 | 3.7759 | 0.3700 | 5 |
+| `static_dropout_0.26` | 0.26 | 4.1784 | 3.9775 | 0.2008 | 5 |
+| `static_dropout_0` | 0.00 | 4.1835 | 3.7750 | 0.4085 | 5 |
+| `static_dropout_0.3` | 0.30 | 4.1946 | 4.0127 | 0.1819 | 5 |

runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525/config.json

@@ -0,0 +1,199 @@
+{
+  "args": {
+    "mode": "locked_stream",
+    "corpus": null,
+    "corpus_glob": null,
+    "text_column": "text",
+    "use_cached_data": true,
+    "output_dir": "runs/wikitext103_l12_streaming_validation_5seed",
+    "resume_from": null,
+    "cache_dir": ".cache/dropout_decay_wikitext103",
+    "models": [
+      "L12_H8_D320=12x8x320"
+    ],
+    "seeds": [
+      1,
+      2,
+      3,
+      4,
+      5
+    ],
+    "token_limits": [
+      5000000
+    ],
+    "stream_token_caps": [
+      250000,
+      500000,
+      1000000,
+      2000000,
+      4000000
+    ],
+    "val_tokens": 500000,
+    "allow_short_corpus": false,
+    "force_retokenize": false,
+    "vocab_size": 4096,
+    "tokenizer_train_chars": 10000000,
+    "block_size": 128,
+    "batch_size": 16,
+    "steps": 2000,
+    "stage_steps": 1000,
+    "dropout_rates": [
+      0.0,
+      0.02,
+      0.04,
+      0.06,
+      0.08,
+      0.1,
+      0.14,
+      0.18,
+      0.2,
+      0.26,
+      0.3
+    ],
+    "decays": [],
+    "anchor_decays": [
+      {
+        "name": "wikitext103_formula_l12",
+        "kind": "anchor_decay",
+        "initial": 0.3,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.3
+          ],
+          [
+            500000,
+            0.26
+          ],
+          [
+            1000000,
+            0.18
+          ],
+          [
+            2000000,
+            0.09
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      },
+      {
+        "name": "wikitext103_probe_blend",
+        "kind": "anchor_decay",
+        "initial": 0.19,
+        "final": 0.01,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.19
+          ],
+          [
+            500000,
+            0.14
+          ],
+          [
+            1000000,
+            0.09
+          ],
+          [
+            2000000,
+            0.04
+          ],
+          [
+            4000000,
+            0.01
+          ]
+        ]
+      },
+      {
+        "name": "wikitext103_low_decay",
+        "kind": "anchor_decay",
+        "initial": 0.14,
+        "final": 0.02,
+        "schedule": "log_prefix_anchor",
+        "decay_tokens": null,
+        "anchors": [
+          [
+            250000,
+            0.14
+          ],
+          [
+            500000,
+            0.14
+          ],
+          [
+            1000000,
+            0.1
+          ],
+          [
+            2000000,
+            0.06
+          ],
+          [
+            4000000,
+            0.02
+          ]
+        ]
+      }
+    ],
+    "decay_tokens": null,
+    "eval_batches": 64,
+    "train_eval_batches": 32,
+    "trace_eval_batches": 8,
+    "eval_every": 0,
+    "log_every": 250,
+    "lr": 0.0003,
+    "weight_decay": 0.1,
+    "grad_clip": 1.0,
+    "plateau_delta": 0.01,
+    "target_min_dropout": 0.1,
+    "min_nonzero_margin": 0.01,
+    "min_high_dropout_margin": 0.03,
+    "screen_early_stop": false,
+    "screen_prune_patience": 3,
+    "screen_prune_min_delta": 0.01
+  },
+  "mode": "locked_stream",
+  "seeds": [
+    1,
+    2,
+    3,
+    4,
+    5
+  ],
+  "models": [
+    {
+      "model_name": "L12_H8_D320",
+      "n_layer": 12,
+      "n_head": 8,
+      "n_embd": 320
+    }
+  ],
+  "device": "mps",
+  "torch": "2.12.0",
+  "python": "3.11.15 (main, Mar  3 2026, 00:52:57) [Clang 21.0.0 (clang-2100.0.123.102)]",
+  "mps_available": true,
+  "attribution": "Derived from Andrej Karpathy's nanochat project (https://github.com/karpathy/nanochat), MIT License, Copyright (c) 2025 Andrej Karpathy.",
+  "tokenizer_path": ".cache/dropout_decay_wikitext103/tokenizer-v4096.json",
+  "encoded_path": ".cache/dropout_decay_wikitext103/tokens-v4096-uint16.npy",
+  "train_tokens": 4500020,
+  "val_tokens": 500000,
+  "effective_token_limits": [
+    4500020
+  ],
+  "effective_stream_token_caps": [
+    250000,
+    500000,
+    1000000,
+    2000000,
+    4000000
+  ],
+  "resume_from": null
+}

runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525/summary.csv

@@ -0,0 +1,71 @@
+run_mode,condition,condition_kind,stage,token_limit,model_name,n_layer,n_head,n_embd,parameters,dropout_initial,dropout_final,dropout_schedule,n,mean_train_eval_loss,std_train_eval_loss,mean_val_eval_loss,std_val_eval_loss,mean_generalization_gap,std_generalization_gap
+locked_stream,static_dropout_0,static,0,250000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,5,3.5230478435754775,0.03634617773629946,5.340271946787834,0.02699193219713777,1.8172241032123566,0.05178790589031804
+locked_stream,static_dropout_0.02,static,0,250000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,5,3.572465108335018,0.027061999609254494,5.274957031011581,0.025475383539799955,1.7024919226765634,0.03656967535619302
+locked_stream,static_dropout_0.04,static,0,250000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,5,3.6440532103180887,0.031242522590923742,5.237822580337524,0.018592151453448197,1.593769370019436,0.03528512909650908
+locked_stream,static_dropout_0.06,static,0,250000,L12_H8_D320,12,8,320,17367040,0.06,0.06,constant,5,3.712760145962238,0.018437630763482243,5.215402702987194,0.01734079554543704,1.5026425570249557,0.03154944432034942
+locked_stream,static_dropout_0.08,static,0,250000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,5,3.761947725713253,0.017361993699706244,5.189354091882706,0.016114118388750917,1.4274063661694527,0.012218707428925264
+locked_stream,static_dropout_0.1,static,0,250000,L12_H8_D320,12,8,320,17367040,0.1,0.1,constant,5,3.8219083666801454,0.019248328397722874,5.169893845915794,0.023739040366881205,1.3479854792356492,0.03169483270954973
+locked_stream,static_dropout_0.14,static,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,5,3.9050611019134522,0.013283976741307785,5.1635470792651175,0.021953339409384615,1.2584859773516655,0.025343683677925957
+locked_stream,static_dropout_0.18,static,0,250000,L12_H8_D320,12,8,320,17367040,0.18,0.18,constant,5,3.9964362382888794,0.01862825818296394,5.16162400841713,0.014987630048664583,1.16518777012825,0.02013674124988852
+locked_stream,static_dropout_0.2,static,0,250000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,5,4.036272630095482,0.013668609797313208,5.170051643252373,0.014071496005973716,1.1337790131568908,0.013715020568700555
+locked_stream,static_dropout_0.26,static,0,250000,L12_H8_D320,12,8,320,17367040,0.26,0.26,constant,5,4.149615630507469,0.024675006962846267,5.194025552272796,0.016128282325428966,1.0444099217653275,0.022196467420881313
+locked_stream,static_dropout_0.3,static,0,250000,L12_H8_D320,12,8,320,17367040,0.3,0.3,constant,5,4.213146212697029,0.013218280901902867,5.214832927286625,0.018060234410376495,1.0016867145895958,0.02723842931339789
+locked_stream,wikitext103_formula_l12,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.3,0.02,log_prefix_anchor,5,4.213146224617958,0.013218263380840874,5.21483291387558,0.018060268226475495,1.0016866892576217,0.027238437205160136
+locked_stream,wikitext103_low_decay,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,5,3.9050611212849615,0.01328398150034886,5.163547059893608,0.02195330299872749,1.2584859386086464,0.025343680837745006
+locked_stream,wikitext103_probe_blend,anchor_decay,0,250000,L12_H8_D320,12,8,320,17367040,0.19,0.01,log_prefix_anchor,5,4.020059056580067,0.016213485710936643,5.165890334546566,0.017075109484847858,1.1458312779664994,0.02016518864116932
+locked_stream,static_dropout_0,static,1,500000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,5,3.1505795806646346,0.025699619025332174,5.174065832793713,0.025168511150111032,2.0234862521290777,0.0236535174506497
+locked_stream,static_dropout_0.02,static,1,500000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,5,3.261230443418026,0.014168838694048941,5.045114178955555,0.016894569246234076,1.783883735537529,0.019840865900389654
+locked_stream,static_dropout_0.04,static,1,500000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,5,3.3515226930379867,0.029537749484816224,4.957330641150475,0.02503345339488141,1.6058079481124878,0.0202359151612378
+locked_stream,static_dropout_0.06,static,1,500000,L12_H8_D320,12,8,320,17367040,0.06,0.06,constant,5,3.4394222095608713,0.013418585935883633,4.894671627879143,0.021331675814859084,1.4552494183182716,0.01946001354297248
+locked_stream,static_dropout_0.08,static,1,500000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,5,3.50356438010931,0.024856947856236954,4.857585413753986,0.023903772104159438,1.3540210336446763,0.0310369653543579
+locked_stream,static_dropout_0.1,static,1,500000,L12_H8_D320,12,8,320,17367040,0.1,0.1,constant,5,3.5636927232146265,0.024862277351108786,4.83315060287714,0.025840187319246186,1.2694578796625138,0.01835928668354016
+locked_stream,static_dropout_0.14,static,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,5,3.6649743527173997,0.023120394246437293,4.800087215006352,0.019846558873074828,1.135112862288952,0.012980966896977213
+locked_stream,static_dropout_0.18,static,1,500000,L12_H8_D320,12,8,320,17367040,0.18,0.18,constant,5,3.7571836963295935,0.024732309568709827,4.794638857245445,0.02062163611688947,1.0374551609158515,0.02312552092553772
+locked_stream,static_dropout_0.2,static,1,500000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,5,3.7914333969354628,0.018987144953080883,4.787301687896251,0.023601504941398347,0.9958682909607888,0.014572669955232898
+locked_stream,static_dropout_0.26,static,1,500000,L12_H8_D320,12,8,320,17367040,0.26,0.26,constant,5,3.905269515514374,0.02376505939443177,4.8081169500947,0.021576250073037834,0.9028474345803261,0.018700407757224247
+locked_stream,static_dropout_0.3,static,1,500000,L12_H8_D320,12,8,320,17367040,0.3,0.3,constant,5,3.976513123512268,0.010796527385978102,4.82416096329689,0.029627048936626245,0.8476478397846222,0.026893359245285242
+locked_stream,wikitext103_formula_l12,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.3,0.02,log_prefix_anchor,5,3.918210099637508,0.011987444359375697,4.80534844994545,0.027829930328897787,0.8871383503079414,0.02626272167699895
+locked_stream,wikitext103_low_decay,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,5,3.6649743929505347,0.023120354646606448,4.800087235867977,0.01984656043518159,1.1351128429174424,0.01298098541252223
+locked_stream,wikitext103_probe_blend,anchor_decay,1,500000,L12_H8_D320,12,8,320,17367040,0.19,0.01,log_prefix_anchor,5,3.684571087360382,0.02397149156090502,4.7872447416186334,0.0268838649061173,1.1026736542582511,0.021535214146850536
+locked_stream,static_dropout_0,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,5,3.3657630145549775,0.025194332688414405,4.766069588065148,0.03242686059194632,1.40030657351017,0.052237334934698405
+locked_stream,static_dropout_0.02,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,5,3.4323942199349404,0.018124036007212796,4.655830132961273,0.01585338982667534,1.2234359130263328,0.024146051923035036
+locked_stream,static_dropout_0.04,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,5,3.502993068099022,0.017291674978225223,4.595940679311752,0.014619476464690098,1.0929476112127303,0.028916946275823
+locked_stream,static_dropout_0.06,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.06,0.06,constant,5,3.555374290049076,0.018874416206200166,4.557375983893872,0.012636266859277006,1.0020016938447953,0.030341255269329737
+locked_stream,static_dropout_0.08,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,5,3.5999757796525955,0.009078845784580064,4.532606348395348,0.011746366682868653,0.9326305687427521,0.019403758262432406
+locked_stream,static_dropout_0.1,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.1,0.1,constant,5,3.652407945692539,0.022251919818600654,4.518611620366573,0.01351751181040844,0.8662036746740341,0.033766642849086155
+locked_stream,static_dropout_0.14,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,5,3.716250878572464,0.02407671653248301,4.500055414438248,0.014819407090300617,0.7838045358657837,0.038101035301189086
+locked_stream,static_dropout_0.18,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.18,0.18,constant,5,3.786629955470562,0.015769997442310384,4.5023036181926726,0.018490102933264013,0.7156736627221107,0.03180040950921459
+locked_stream,static_dropout_0.2,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,5,3.814779002964497,0.01587114098616348,4.506029562652111,0.02043622151331783,0.6912505596876144,0.03147408021666257
+locked_stream,static_dropout_0.26,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.26,0.26,constant,5,3.9071271896362303,0.02774032700763299,4.52620010226965,0.010149133513914643,0.619072912633419,0.035195638721522884
+locked_stream,static_dropout_0.3,static,2,1000000,L12_H8_D320,12,8,320,17367040,0.3,0.3,constant,5,3.970817744731903,0.017137273586986676,4.546238152682781,0.012672630425565377,0.5754204079508781,0.02420747470801454
+locked_stream,wikitext103_formula_l12,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.3,0.02,log_prefix_anchor,5,3.8283171117305757,0.017535628899982475,4.49377838075161,0.014674108647494532,0.6654612690210342,0.02707089996550965
+locked_stream,wikitext103_low_decay,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,5,3.6606912568211554,0.023899445408052746,4.50133255571127,0.01578619433972172,0.8406412988901139,0.03886870429434243
+locked_stream,wikitext103_probe_blend,anchor_decay,2,1000000,L12_H8_D320,12,8,320,17367040,0.19,0.01,log_prefix_anchor,5,3.6495467707514764,0.019490464893964263,4.49399798810482,0.015864095933133645,0.8444512173533439,0.03475725994195922
+locked_stream,static_dropout_0,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,5,3.6409011498093604,0.014124132971310668,4.417352265119552,0.01876483626309062,0.7764511153101921,0.024211298020746064
+locked_stream,static_dropout_0.02,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,5,3.6672953754663467,0.010966930251295475,4.355083760619164,0.026489436313039863,0.6877883851528168,0.026106538952549097
+locked_stream,static_dropout_0.04,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,5,3.703781445324421,0.01970997472996272,4.3262467920780185,0.016533384756725625,0.6224653467535972,0.02577206799431021
+locked_stream,static_dropout_0.06,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.06,0.06,constant,5,3.7317687764763834,0.017157479537739013,4.305900506675243,0.018234543669842573,0.5741317301988602,0.023633182905307998
+locked_stream,static_dropout_0.08,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,5,3.7518652200698854,0.019831550751639896,4.292461846768856,0.016558424281915483,0.5405966266989708,0.034040566494281235
+locked_stream,static_dropout_0.1,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.1,0.1,constant,5,3.7808884128928186,0.01766528157534878,4.285703422129155,0.018994111876547076,0.5048150092363357,0.028072111875115907
+locked_stream,static_dropout_0.14,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,5,3.828673002123833,0.004937581309037719,4.281563547253609,0.02138858846966249,0.452890545129776,0.01755457638984346
+locked_stream,static_dropout_0.18,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.18,0.18,constant,5,3.876796562969685,0.008357602777023191,4.288915157318115,0.019532110178580833,0.4121185943484306,0.013904819825939841
+locked_stream,static_dropout_0.2,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,5,3.898232290148735,0.008893270752025355,4.294206875562668,0.016019909189912813,0.3959745854139328,0.01268490098888804
+locked_stream,static_dropout_0.26,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.26,0.26,constant,5,3.9654731526970863,0.018160558354021094,4.32477528527379,0.017871972287011016,0.359302132576704,0.018728321745075875
+locked_stream,static_dropout_0.3,static,3,2000000,L12_H8_D320,12,8,320,17367040,0.3,0.3,constant,5,4.009651578962803,0.014639843511797184,4.34667577445507,0.015854348211857338,0.3370241954922676,0.019075525784314876
+locked_stream,wikitext103_formula_l12,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.3,0.02,log_prefix_anchor,5,3.8088956013321877,0.016376701234484724,4.260725708305836,0.018098739272622868,0.45183010697364806,0.01960955973506382
+locked_stream,wikitext103_low_decay,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,5,3.7474397838115694,0.003830485352807811,4.273568108677864,0.022770462958788148,0.5261283248662949,0.02051193157067845
+locked_stream,wikitext103_probe_blend,anchor_decay,3,2000000,L12_H8_D320,12,8,320,17367040,0.19,0.01,log_prefix_anchor,5,3.720025636255741,0.012889615922098394,4.274319227784872,0.0173783813582558,0.5542935915291309,0.012746502538982045
+locked_stream,static_dropout_0,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.0,0.0,constant,5,3.7750062167644503,0.025453964920340872,4.183486974239349,0.01652781602088682,0.4084807574748993,0.03358628463770181
+locked_stream,static_dropout_0.02,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.02,0.02,constant,5,3.775918734073639,0.03360785862475549,4.145870254188776,0.016455755256661365,0.3699515201151371,0.0368559849747316
+locked_stream,static_dropout_0.04,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.04,0.04,constant,5,3.7978432700037956,0.018504253509089505,4.1331265024840835,0.022679096058077455,0.3352832324802876,0.036956450676459744
+locked_stream,static_dropout_0.06,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.06,0.06,constant,5,3.806561966240406,0.022924307415498075,4.119682380557061,0.008179605355252681,0.3131204143166542,0.028667268842643943
+locked_stream,static_dropout_0.08,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.08,0.08,constant,5,3.8268220275640488,0.021203534366681782,4.111632210761309,0.01857117977689519,0.2848101831972599,0.03679859980086126
+locked_stream,static_dropout_0.1,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.1,0.1,constant,5,3.841722105443478,0.019884552697005082,4.1104619488120075,0.018806640195540816,0.2687398433685303,0.03219305563944893
+locked_stream,static_dropout_0.14,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.14,constant,5,3.8673695802688597,0.025363134860510252,4.122133788466454,0.015529540565590345,0.2547642081975937,0.0302072467927019
+locked_stream,static_dropout_0.18,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.18,0.18,constant,5,3.9014849200844766,0.018144303418082167,4.130411610752344,0.013005735782293457,0.22892669066786767,0.02966442514236336
+locked_stream,static_dropout_0.2,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.2,0.2,constant,5,3.9154964342713354,0.020116706552102667,4.1393771633505825,0.016742174044810716,0.22388072907924653,0.033233695598367155
+locked_stream,static_dropout_0.26,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.26,0.26,constant,5,3.9775424867868425,0.021977989750098077,4.178351600468159,0.01453281035529064,0.20080911368131638,0.03295020359909027
+locked_stream,static_dropout_0.3,static,4,4000000,L12_H8_D320,12,8,320,17367040,0.3,0.3,constant,5,4.012722708284855,0.024177126067976527,4.194609892368317,0.014131822174274241,0.18188718408346177,0.031100561007876955
+locked_stream,wikitext103_formula_l12,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.3,0.02,log_prefix_anchor,5,3.7991417959332465,0.028273745467366926,4.080797865241766,0.019458062276034267,0.2816560693085194,0.038849358858741
+locked_stream,wikitext103_low_decay,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.14,0.02,log_prefix_anchor,5,3.776866267621517,0.02647416884246062,4.102013133466244,0.016554390946653275,0.3251468658447266,0.03314100249583806
+locked_stream,wikitext103_probe_blend,anchor_decay,4,4000000,L12_H8_D320,12,8,320,17367040,0.19,0.01,log_prefix_anchor,5,3.7674024373292925,0.027214408565503653,4.096089626103639,0.014523730760550201,0.3286871887743473,0.038218075738052845

runs/wikitext103_l12_streaming_validation_5seed/locked_stream/20260531-093525/summary.json

@@ -0,0 +1,1542 @@
+[
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.5230478435754775,
+    "std_train_eval_loss": 0.03634617773629946,
+    "mean_val_eval_loss": 5.340271946787834,
+    "std_val_eval_loss": 0.02699193219713777,
+    "mean_generalization_gap": 1.8172241032123566,
+    "std_generalization_gap": 0.05178790589031804
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.572465108335018,
+    "std_train_eval_loss": 0.027061999609254494,
+    "mean_val_eval_loss": 5.274957031011581,
+    "std_val_eval_loss": 0.025475383539799955,
+    "mean_generalization_gap": 1.7024919226765634,
+    "std_generalization_gap": 0.03656967535619302
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.6440532103180887,
+    "std_train_eval_loss": 0.031242522590923742,
+    "mean_val_eval_loss": 5.237822580337524,
+    "std_val_eval_loss": 0.018592151453448197,
+    "mean_generalization_gap": 1.593769370019436,
+    "std_generalization_gap": 0.03528512909650908
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.06",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.06,
+    "dropout_final": 0.06,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.712760145962238,
+    "std_train_eval_loss": 0.018437630763482243,
+    "mean_val_eval_loss": 5.215402702987194,
+    "std_val_eval_loss": 0.01734079554543704,
+    "mean_generalization_gap": 1.5026425570249557,
+    "std_generalization_gap": 0.03154944432034942
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.761947725713253,
+    "std_train_eval_loss": 0.017361993699706244,
+    "mean_val_eval_loss": 5.189354091882706,
+    "std_val_eval_loss": 0.016114118388750917,
+    "mean_generalization_gap": 1.4274063661694527,
+    "std_generalization_gap": 0.012218707428925264
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.1",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.1,
+    "dropout_final": 0.1,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.8219083666801454,
+    "std_train_eval_loss": 0.019248328397722874,
+    "mean_val_eval_loss": 5.169893845915794,
+    "std_val_eval_loss": 0.023739040366881205,
+    "mean_generalization_gap": 1.3479854792356492,
+    "std_generalization_gap": 0.03169483270954973
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9050611019134522,
+    "std_train_eval_loss": 0.013283976741307785,
+    "mean_val_eval_loss": 5.1635470792651175,
+    "std_val_eval_loss": 0.021953339409384615,
+    "mean_generalization_gap": 1.2584859773516655,
+    "std_generalization_gap": 0.025343683677925957
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.18",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.18,
+    "dropout_final": 0.18,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9964362382888794,
+    "std_train_eval_loss": 0.01862825818296394,
+    "mean_val_eval_loss": 5.16162400841713,
+    "std_val_eval_loss": 0.014987630048664583,
+    "mean_generalization_gap": 1.16518777012825,
+    "std_generalization_gap": 0.02013674124988852
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 4.036272630095482,
+    "std_train_eval_loss": 0.013668609797313208,
+    "mean_val_eval_loss": 5.170051643252373,
+    "std_val_eval_loss": 0.014071496005973716,
+    "mean_generalization_gap": 1.1337790131568908,
+    "std_generalization_gap": 0.013715020568700555
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.26",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.26,
+    "dropout_final": 0.26,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 4.149615630507469,
+    "std_train_eval_loss": 0.024675006962846267,
+    "mean_val_eval_loss": 5.194025552272796,
+    "std_val_eval_loss": 0.016128282325428966,
+    "mean_generalization_gap": 1.0444099217653275,
+    "std_generalization_gap": 0.022196467420881313
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.3",
+    "condition_kind": "static",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.3,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 4.213146212697029,
+    "std_train_eval_loss": 0.013218280901902867,
+    "mean_val_eval_loss": 5.214832927286625,
+    "std_val_eval_loss": 0.018060234410376495,
+    "mean_generalization_gap": 1.0016867145895958,
+    "std_generalization_gap": 0.02723842931339789
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 4.213146224617958,
+    "std_train_eval_loss": 0.013218263380840874,
+    "mean_val_eval_loss": 5.21483291387558,
+    "std_val_eval_loss": 0.018060268226475495,
+    "mean_generalization_gap": 1.0016866892576217,
+    "std_generalization_gap": 0.027238437205160136
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_low_decay",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.9050611212849615,
+    "std_train_eval_loss": 0.01328398150034886,
+    "mean_val_eval_loss": 5.163547059893608,
+    "std_val_eval_loss": 0.02195330299872749,
+    "mean_generalization_gap": 1.2584859386086464,
+    "std_generalization_gap": 0.025343680837745006
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_probe_blend",
+    "condition_kind": "anchor_decay",
+    "stage": 0,
+    "token_limit": 250000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.19,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 4.020059056580067,
+    "std_train_eval_loss": 0.016213485710936643,
+    "mean_val_eval_loss": 5.165890334546566,
+    "std_val_eval_loss": 0.017075109484847858,
+    "mean_generalization_gap": 1.1458312779664994,
+    "std_generalization_gap": 0.02016518864116932
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.1505795806646346,
+    "std_train_eval_loss": 0.025699619025332174,
+    "mean_val_eval_loss": 5.174065832793713,
+    "std_val_eval_loss": 0.025168511150111032,
+    "mean_generalization_gap": 2.0234862521290777,
+    "std_generalization_gap": 0.0236535174506497
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.261230443418026,
+    "std_train_eval_loss": 0.014168838694048941,
+    "mean_val_eval_loss": 5.045114178955555,
+    "std_val_eval_loss": 0.016894569246234076,
+    "mean_generalization_gap": 1.783883735537529,
+    "std_generalization_gap": 0.019840865900389654
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.3515226930379867,
+    "std_train_eval_loss": 0.029537749484816224,
+    "mean_val_eval_loss": 4.957330641150475,
+    "std_val_eval_loss": 0.02503345339488141,
+    "mean_generalization_gap": 1.6058079481124878,
+    "std_generalization_gap": 0.0202359151612378
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.06",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.06,
+    "dropout_final": 0.06,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.4394222095608713,
+    "std_train_eval_loss": 0.013418585935883633,
+    "mean_val_eval_loss": 4.894671627879143,
+    "std_val_eval_loss": 0.021331675814859084,
+    "mean_generalization_gap": 1.4552494183182716,
+    "std_generalization_gap": 0.01946001354297248
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.50356438010931,
+    "std_train_eval_loss": 0.024856947856236954,
+    "mean_val_eval_loss": 4.857585413753986,
+    "std_val_eval_loss": 0.023903772104159438,
+    "mean_generalization_gap": 1.3540210336446763,
+    "std_generalization_gap": 0.0310369653543579
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.1",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.1,
+    "dropout_final": 0.1,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.5636927232146265,
+    "std_train_eval_loss": 0.024862277351108786,
+    "mean_val_eval_loss": 4.83315060287714,
+    "std_val_eval_loss": 0.025840187319246186,
+    "mean_generalization_gap": 1.2694578796625138,
+    "std_generalization_gap": 0.01835928668354016
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.6649743527173997,
+    "std_train_eval_loss": 0.023120394246437293,
+    "mean_val_eval_loss": 4.800087215006352,
+    "std_val_eval_loss": 0.019846558873074828,
+    "mean_generalization_gap": 1.135112862288952,
+    "std_generalization_gap": 0.012980966896977213
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.18",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.18,
+    "dropout_final": 0.18,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7571836963295935,
+    "std_train_eval_loss": 0.024732309568709827,
+    "mean_val_eval_loss": 4.794638857245445,
+    "std_val_eval_loss": 0.02062163611688947,
+    "mean_generalization_gap": 1.0374551609158515,
+    "std_generalization_gap": 0.02312552092553772
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7914333969354628,
+    "std_train_eval_loss": 0.018987144953080883,
+    "mean_val_eval_loss": 4.787301687896251,
+    "std_val_eval_loss": 0.023601504941398347,
+    "mean_generalization_gap": 0.9958682909607888,
+    "std_generalization_gap": 0.014572669955232898
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.26",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.26,
+    "dropout_final": 0.26,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.905269515514374,
+    "std_train_eval_loss": 0.02376505939443177,
+    "mean_val_eval_loss": 4.8081169500947,
+    "std_val_eval_loss": 0.021576250073037834,
+    "mean_generalization_gap": 0.9028474345803261,
+    "std_generalization_gap": 0.018700407757224247
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.3",
+    "condition_kind": "static",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.3,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.976513123512268,
+    "std_train_eval_loss": 0.010796527385978102,
+    "mean_val_eval_loss": 4.82416096329689,
+    "std_val_eval_loss": 0.029627048936626245,
+    "mean_generalization_gap": 0.8476478397846222,
+    "std_generalization_gap": 0.026893359245285242
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.918210099637508,
+    "std_train_eval_loss": 0.011987444359375697,
+    "mean_val_eval_loss": 4.80534844994545,
+    "std_val_eval_loss": 0.027829930328897787,
+    "mean_generalization_gap": 0.8871383503079414,
+    "std_generalization_gap": 0.02626272167699895
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_low_decay",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.6649743929505347,
+    "std_train_eval_loss": 0.023120354646606448,
+    "mean_val_eval_loss": 4.800087235867977,
+    "std_val_eval_loss": 0.01984656043518159,
+    "mean_generalization_gap": 1.1351128429174424,
+    "std_generalization_gap": 0.01298098541252223
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_probe_blend",
+    "condition_kind": "anchor_decay",
+    "stage": 1,
+    "token_limit": 500000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.19,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.684571087360382,
+    "std_train_eval_loss": 0.02397149156090502,
+    "mean_val_eval_loss": 4.7872447416186334,
+    "std_val_eval_loss": 0.0268838649061173,
+    "mean_generalization_gap": 1.1026736542582511,
+    "std_generalization_gap": 0.021535214146850536
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.3657630145549775,
+    "std_train_eval_loss": 0.025194332688414405,
+    "mean_val_eval_loss": 4.766069588065148,
+    "std_val_eval_loss": 0.03242686059194632,
+    "mean_generalization_gap": 1.40030657351017,
+    "std_generalization_gap": 0.052237334934698405
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.4323942199349404,
+    "std_train_eval_loss": 0.018124036007212796,
+    "mean_val_eval_loss": 4.655830132961273,
+    "std_val_eval_loss": 0.01585338982667534,
+    "mean_generalization_gap": 1.2234359130263328,
+    "std_generalization_gap": 0.024146051923035036
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.502993068099022,
+    "std_train_eval_loss": 0.017291674978225223,
+    "mean_val_eval_loss": 4.595940679311752,
+    "std_val_eval_loss": 0.014619476464690098,
+    "mean_generalization_gap": 1.0929476112127303,
+    "std_generalization_gap": 0.028916946275823
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.06",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.06,
+    "dropout_final": 0.06,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.555374290049076,
+    "std_train_eval_loss": 0.018874416206200166,
+    "mean_val_eval_loss": 4.557375983893872,
+    "std_val_eval_loss": 0.012636266859277006,
+    "mean_generalization_gap": 1.0020016938447953,
+    "std_generalization_gap": 0.030341255269329737
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.5999757796525955,
+    "std_train_eval_loss": 0.009078845784580064,
+    "mean_val_eval_loss": 4.532606348395348,
+    "std_val_eval_loss": 0.011746366682868653,
+    "mean_generalization_gap": 0.9326305687427521,
+    "std_generalization_gap": 0.019403758262432406
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.1",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.1,
+    "dropout_final": 0.1,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.652407945692539,
+    "std_train_eval_loss": 0.022251919818600654,
+    "mean_val_eval_loss": 4.518611620366573,
+    "std_val_eval_loss": 0.01351751181040844,
+    "mean_generalization_gap": 0.8662036746740341,
+    "std_generalization_gap": 0.033766642849086155
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.716250878572464,
+    "std_train_eval_loss": 0.02407671653248301,
+    "mean_val_eval_loss": 4.500055414438248,
+    "std_val_eval_loss": 0.014819407090300617,
+    "mean_generalization_gap": 0.7838045358657837,
+    "std_generalization_gap": 0.038101035301189086
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.18",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.18,
+    "dropout_final": 0.18,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.786629955470562,
+    "std_train_eval_loss": 0.015769997442310384,
+    "mean_val_eval_loss": 4.5023036181926726,
+    "std_val_eval_loss": 0.018490102933264013,
+    "mean_generalization_gap": 0.7156736627221107,
+    "std_generalization_gap": 0.03180040950921459
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.814779002964497,
+    "std_train_eval_loss": 0.01587114098616348,
+    "mean_val_eval_loss": 4.506029562652111,
+    "std_val_eval_loss": 0.02043622151331783,
+    "mean_generalization_gap": 0.6912505596876144,
+    "std_generalization_gap": 0.03147408021666257
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.26",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.26,
+    "dropout_final": 0.26,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9071271896362303,
+    "std_train_eval_loss": 0.02774032700763299,
+    "mean_val_eval_loss": 4.52620010226965,
+    "std_val_eval_loss": 0.010149133513914643,
+    "mean_generalization_gap": 0.619072912633419,
+    "std_generalization_gap": 0.035195638721522884
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.3",
+    "condition_kind": "static",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.3,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.970817744731903,
+    "std_train_eval_loss": 0.017137273586986676,
+    "mean_val_eval_loss": 4.546238152682781,
+    "std_val_eval_loss": 0.012672630425565377,
+    "mean_generalization_gap": 0.5754204079508781,
+    "std_generalization_gap": 0.02420747470801454
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.8283171117305757,
+    "std_train_eval_loss": 0.017535628899982475,
+    "mean_val_eval_loss": 4.49377838075161,
+    "std_val_eval_loss": 0.014674108647494532,
+    "mean_generalization_gap": 0.6654612690210342,
+    "std_generalization_gap": 0.02707089996550965
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_low_decay",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.6606912568211554,
+    "std_train_eval_loss": 0.023899445408052746,
+    "mean_val_eval_loss": 4.50133255571127,
+    "std_val_eval_loss": 0.01578619433972172,
+    "mean_generalization_gap": 0.8406412988901139,
+    "std_generalization_gap": 0.03886870429434243
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_probe_blend",
+    "condition_kind": "anchor_decay",
+    "stage": 2,
+    "token_limit": 1000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.19,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.6495467707514764,
+    "std_train_eval_loss": 0.019490464893964263,
+    "mean_val_eval_loss": 4.49399798810482,
+    "std_val_eval_loss": 0.015864095933133645,
+    "mean_generalization_gap": 0.8444512173533439,
+    "std_generalization_gap": 0.03475725994195922
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.6409011498093604,
+    "std_train_eval_loss": 0.014124132971310668,
+    "mean_val_eval_loss": 4.417352265119552,
+    "std_val_eval_loss": 0.01876483626309062,
+    "mean_generalization_gap": 0.7764511153101921,
+    "std_generalization_gap": 0.024211298020746064
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.6672953754663467,
+    "std_train_eval_loss": 0.010966930251295475,
+    "mean_val_eval_loss": 4.355083760619164,
+    "std_val_eval_loss": 0.026489436313039863,
+    "mean_generalization_gap": 0.6877883851528168,
+    "std_generalization_gap": 0.026106538952549097
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.703781445324421,
+    "std_train_eval_loss": 0.01970997472996272,
+    "mean_val_eval_loss": 4.3262467920780185,
+    "std_val_eval_loss": 0.016533384756725625,
+    "mean_generalization_gap": 0.6224653467535972,
+    "std_generalization_gap": 0.02577206799431021
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.06",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.06,
+    "dropout_final": 0.06,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7317687764763834,
+    "std_train_eval_loss": 0.017157479537739013,
+    "mean_val_eval_loss": 4.305900506675243,
+    "std_val_eval_loss": 0.018234543669842573,
+    "mean_generalization_gap": 0.5741317301988602,
+    "std_generalization_gap": 0.023633182905307998
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7518652200698854,
+    "std_train_eval_loss": 0.019831550751639896,
+    "mean_val_eval_loss": 4.292461846768856,
+    "std_val_eval_loss": 0.016558424281915483,
+    "mean_generalization_gap": 0.5405966266989708,
+    "std_generalization_gap": 0.034040566494281235
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.1",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.1,
+    "dropout_final": 0.1,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7808884128928186,
+    "std_train_eval_loss": 0.01766528157534878,
+    "mean_val_eval_loss": 4.285703422129155,
+    "std_val_eval_loss": 0.018994111876547076,
+    "mean_generalization_gap": 0.5048150092363357,
+    "std_generalization_gap": 0.028072111875115907
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.828673002123833,
+    "std_train_eval_loss": 0.004937581309037719,
+    "mean_val_eval_loss": 4.281563547253609,
+    "std_val_eval_loss": 0.02138858846966249,
+    "mean_generalization_gap": 0.452890545129776,
+    "std_generalization_gap": 0.01755457638984346
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.18",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.18,
+    "dropout_final": 0.18,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.876796562969685,
+    "std_train_eval_loss": 0.008357602777023191,
+    "mean_val_eval_loss": 4.288915157318115,
+    "std_val_eval_loss": 0.019532110178580833,
+    "mean_generalization_gap": 0.4121185943484306,
+    "std_generalization_gap": 0.013904819825939841
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.898232290148735,
+    "std_train_eval_loss": 0.008893270752025355,
+    "mean_val_eval_loss": 4.294206875562668,
+    "std_val_eval_loss": 0.016019909189912813,
+    "mean_generalization_gap": 0.3959745854139328,
+    "std_generalization_gap": 0.01268490098888804
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.26",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.26,
+    "dropout_final": 0.26,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9654731526970863,
+    "std_train_eval_loss": 0.018160558354021094,
+    "mean_val_eval_loss": 4.32477528527379,
+    "std_val_eval_loss": 0.017871972287011016,
+    "mean_generalization_gap": 0.359302132576704,
+    "std_generalization_gap": 0.018728321745075875
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.3",
+    "condition_kind": "static",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.3,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 4.009651578962803,
+    "std_train_eval_loss": 0.014639843511797184,
+    "mean_val_eval_loss": 4.34667577445507,
+    "std_val_eval_loss": 0.015854348211857338,
+    "mean_generalization_gap": 0.3370241954922676,
+    "std_generalization_gap": 0.019075525784314876
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.8088956013321877,
+    "std_train_eval_loss": 0.016376701234484724,
+    "mean_val_eval_loss": 4.260725708305836,
+    "std_val_eval_loss": 0.018098739272622868,
+    "mean_generalization_gap": 0.45183010697364806,
+    "std_generalization_gap": 0.01960955973506382
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_low_decay",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.7474397838115694,
+    "std_train_eval_loss": 0.003830485352807811,
+    "mean_val_eval_loss": 4.273568108677864,
+    "std_val_eval_loss": 0.022770462958788148,
+    "mean_generalization_gap": 0.5261283248662949,
+    "std_generalization_gap": 0.02051193157067845
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_probe_blend",
+    "condition_kind": "anchor_decay",
+    "stage": 3,
+    "token_limit": 2000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.19,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.720025636255741,
+    "std_train_eval_loss": 0.012889615922098394,
+    "mean_val_eval_loss": 4.274319227784872,
+    "std_val_eval_loss": 0.0173783813582558,
+    "mean_generalization_gap": 0.5542935915291309,
+    "std_generalization_gap": 0.012746502538982045
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.0,
+    "dropout_final": 0.0,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7750062167644503,
+    "std_train_eval_loss": 0.025453964920340872,
+    "mean_val_eval_loss": 4.183486974239349,
+    "std_val_eval_loss": 0.01652781602088682,
+    "mean_generalization_gap": 0.4084807574748993,
+    "std_generalization_gap": 0.03358628463770181
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.02",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.02,
+    "dropout_final": 0.02,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.775918734073639,
+    "std_train_eval_loss": 0.03360785862475549,
+    "mean_val_eval_loss": 4.145870254188776,
+    "std_val_eval_loss": 0.016455755256661365,
+    "mean_generalization_gap": 0.3699515201151371,
+    "std_generalization_gap": 0.0368559849747316
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.04",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.04,
+    "dropout_final": 0.04,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.7978432700037956,
+    "std_train_eval_loss": 0.018504253509089505,
+    "mean_val_eval_loss": 4.1331265024840835,
+    "std_val_eval_loss": 0.022679096058077455,
+    "mean_generalization_gap": 0.3352832324802876,
+    "std_generalization_gap": 0.036956450676459744
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.06",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.06,
+    "dropout_final": 0.06,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.806561966240406,
+    "std_train_eval_loss": 0.022924307415498075,
+    "mean_val_eval_loss": 4.119682380557061,
+    "std_val_eval_loss": 0.008179605355252681,
+    "mean_generalization_gap": 0.3131204143166542,
+    "std_generalization_gap": 0.028667268842643943
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.08",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.08,
+    "dropout_final": 0.08,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.8268220275640488,
+    "std_train_eval_loss": 0.021203534366681782,
+    "mean_val_eval_loss": 4.111632210761309,
+    "std_val_eval_loss": 0.01857117977689519,
+    "mean_generalization_gap": 0.2848101831972599,
+    "std_generalization_gap": 0.03679859980086126
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.1",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.1,
+    "dropout_final": 0.1,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.841722105443478,
+    "std_train_eval_loss": 0.019884552697005082,
+    "mean_val_eval_loss": 4.1104619488120075,
+    "std_val_eval_loss": 0.018806640195540816,
+    "mean_generalization_gap": 0.2687398433685303,
+    "std_generalization_gap": 0.03219305563944893
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.14",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.14,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.8673695802688597,
+    "std_train_eval_loss": 0.025363134860510252,
+    "mean_val_eval_loss": 4.122133788466454,
+    "std_val_eval_loss": 0.015529540565590345,
+    "mean_generalization_gap": 0.2547642081975937,
+    "std_generalization_gap": 0.0302072467927019
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.18",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.18,
+    "dropout_final": 0.18,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9014849200844766,
+    "std_train_eval_loss": 0.018144303418082167,
+    "mean_val_eval_loss": 4.130411610752344,
+    "std_val_eval_loss": 0.013005735782293457,
+    "mean_generalization_gap": 0.22892669066786767,
+    "std_generalization_gap": 0.02966442514236336
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.2",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.2,
+    "dropout_final": 0.2,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9154964342713354,
+    "std_train_eval_loss": 0.020116706552102667,
+    "mean_val_eval_loss": 4.1393771633505825,
+    "std_val_eval_loss": 0.016742174044810716,
+    "mean_generalization_gap": 0.22388072907924653,
+    "std_generalization_gap": 0.033233695598367155
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.26",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.26,
+    "dropout_final": 0.26,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 3.9775424867868425,
+    "std_train_eval_loss": 0.021977989750098077,
+    "mean_val_eval_loss": 4.178351600468159,
+    "std_val_eval_loss": 0.01453281035529064,
+    "mean_generalization_gap": 0.20080911368131638,
+    "std_generalization_gap": 0.03295020359909027
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "static_dropout_0.3",
+    "condition_kind": "static",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.3,
+    "dropout_schedule": "constant",
+    "n": 5,
+    "mean_train_eval_loss": 4.012722708284855,
+    "std_train_eval_loss": 0.024177126067976527,
+    "mean_val_eval_loss": 4.194609892368317,
+    "std_val_eval_loss": 0.014131822174274241,
+    "mean_generalization_gap": 0.18188718408346177,
+    "std_generalization_gap": 0.031100561007876955
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_formula_l12",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.3,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.7991417959332465,
+    "std_train_eval_loss": 0.028273745467366926,
+    "mean_val_eval_loss": 4.080797865241766,
+    "std_val_eval_loss": 0.019458062276034267,
+    "mean_generalization_gap": 0.2816560693085194,
+    "std_generalization_gap": 0.038849358858741
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_low_decay",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.14,
+    "dropout_final": 0.02,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.776866267621517,
+    "std_train_eval_loss": 0.02647416884246062,
+    "mean_val_eval_loss": 4.102013133466244,
+    "std_val_eval_loss": 0.016554390946653275,
+    "mean_generalization_gap": 0.3251468658447266,
+    "std_generalization_gap": 0.03314100249583806
+  },
+  {
+    "run_mode": "locked_stream",
+    "condition": "wikitext103_probe_blend",
+    "condition_kind": "anchor_decay",
+    "stage": 4,
+    "token_limit": 4000000,
+    "model_name": "L12_H8_D320",
+    "n_layer": 12,
+    "n_head": 8,
+    "n_embd": 320,
+    "parameters": 17367040,
+    "dropout_initial": 0.19,
+    "dropout_final": 0.01,
+    "dropout_schedule": "log_prefix_anchor",
+    "n": 5,
+    "mean_train_eval_loss": 3.7674024373292925,
+    "std_train_eval_loss": 0.027214408565503653,
+    "mean_val_eval_loss": 4.096089626103639,
+    "std_val_eval_loss": 0.014523730760550201,
+    "mean_generalization_gap": 0.3286871887743473,
+    "std_generalization_gap": 0.038218075738052845
+  }
+]

runs/wikitext103_streaming_report/l12_validation_5seed/condition_summary.csv

@@ -0,0 +1,15 @@
+condition,kind,n,mean_trajectory_val,std_trajectory_val,mean_final_val,std_final_val,mean_final_gap,std_final_gap,dropout_path
+wikitext103_formula_l12,anchor_decay,5,4.571096663624049,0.004493905823534624,4.080797865241766,0.019458062276034267,0.2816560693085194,0.038849358858741,0.30 -> 0.26 -> 0.18 -> 0.09 -> 0.02
+wikitext103_probe_blend,anchor_decay,5,4.5635083836317065,0.0046223860048219215,4.096089626103639,0.014523730760550201,0.3286871887743473,0.038218075738052845,0.19 -> 0.14 -> 0.09 -> 0.04 -> 0.01
+wikitext103_low_decay,anchor_decay,5,4.568109618723392,0.007251133716114802,4.102013133466244,0.016554390946653275,0.3251468658447266,0.03314100249583806,0.14 -> 0.14 -> 0.10 -> 0.06 -> 0.02
+static_dropout_0.1,static,5,4.583564288020134,0.00624825224116908,4.1104619488120075,0.018806640195540816,0.2687398433685303,0.03219305563944893,0.10 -> 0.10 -> 0.10 -> 0.10 -> 0.10
+static_dropout_0.08,static,5,4.596727982312441,0.007294540304515209,4.111632210761309,0.01857117977689519,0.2848101831972599,0.03679859980086126,0.08 -> 0.08 -> 0.08 -> 0.08 -> 0.08
+static_dropout_0.06,static,5,4.618606640398502,0.004793067320536325,4.119682380557061,0.008179605355252681,0.3131204143166542,0.028667268842643943,0.06 -> 0.06 -> 0.06 -> 0.06 -> 0.06
+static_dropout_0.14,static,5,4.573477408885956,0.007705324015227247,4.122133788466454,0.015529540565590345,0.2547642081975937,0.0302072467927019,0.14 -> 0.14 -> 0.14 -> 0.14 -> 0.14
+static_dropout_0.18,static,5,4.575578650385141,0.004087884073210763,4.130411610752344,0.013005735782293457,0.22892669066786767,0.02966442514236336,0.18 -> 0.18 -> 0.18 -> 0.18 -> 0.18
+static_dropout_0.04,static,5,4.6500934390723705,0.007686314739184788,4.1331265024840835,0.022679096058077455,0.3352832324802876,0.036956450676459744,0.04 -> 0.04 -> 0.04 -> 0.04 -> 0.04
+static_dropout_0.2,static,5,4.579393386542797,0.004961083585861927,4.1393771633505825,0.016742174044810716,0.22388072907924653,0.033233695598367155,0.20 -> 0.20 -> 0.20 -> 0.20 -> 0.20
+static_dropout_0.02,static,5,4.695371071547269,0.008590324433750853,4.145870254188776,0.016455755256661365,0.3699515201151371,0.0368559849747316,0.02 -> 0.02 -> 0.02 -> 0.02 -> 0.02
+static_dropout_0.26,static,5,4.606293898075819,0.0051111340096229975,4.178351600468159,0.01453281035529064,0.20080911368131638,0.03295020359909027,0.26 -> 0.26 -> 0.26 -> 0.26 -> 0.26
+static_dropout_0,static,5,4.776249321401119,0.010887492470564343,4.183486974239349,0.01652781602088682,0.4084807574748993,0.03358628463770181,0.00 -> 0.00 -> 0.00 -> 0.00 -> 0.00
+static_dropout_0.3,static,5,4.625303542017937,0.003399857155492287,4.194609892368317,0.014131822174274241,0.18188718408346177,0.031100561007876955,0.30 -> 0.30 -> 0.30 -> 0.30 -> 0.30

runs/wikitext103_streaming_report/l12_validation_5seed/paired_final_deltas.csv

@@ -0,0 +1,71 @@
+seed,condition,final_val,best_static_condition,best_static_final_val,delta_vs_best_static
+1,wikitext103_formula_l12,4.062299974262714,static_dropout_0.1,4.080718021839857,-0.018418047577142715
+1,wikitext103_probe_blend,4.07382857054472,static_dropout_0.1,4.080718021839857,-0.006889451295137405
+1,wikitext103_low_decay,4.085419669747353,static_dropout_0.1,4.080718021839857,0.004701647907495499
+1,static_dropout_0.1,4.080718021839857,static_dropout_0.1,4.080718021839857,0.0
+1,static_dropout_0.08,4.089278034865856,static_dropout_0.1,4.080718021839857,0.00856001302599907
+1,static_dropout_0.06,4.111171845346689,static_dropout_0.1,4.080718021839857,0.030453823506832123
+1,static_dropout_0.14,4.108215663582087,static_dropout_0.1,4.080718021839857,0.02749764174222946
+1,static_dropout_0.18,4.110849380493164,static_dropout_0.1,4.080718021839857,0.03013135865330696
+1,static_dropout_0.2,4.116236750036478,static_dropout_0.1,4.080718021839857,0.03551872819662094
+1,static_dropout_0.04,4.103101845830679,static_dropout_0.1,4.080718021839857,0.02238382399082184
+1,static_dropout_0.02,4.137094188481569,static_dropout_0.1,4.080718021839857,0.05637616664171219
+1,static_dropout_0,4.159992527216673,static_dropout_0.1,4.080718021839857,0.0792745053768158
+1,static_dropout_0.26,4.155716687440872,static_dropout_0.1,4.080718021839857,0.07499866560101509
+1,static_dropout_0.3,4.180164486169815,static_dropout_0.1,4.080718021839857,0.09944646432995796
+2,wikitext103_formula_l12,4.112260434776545,static_dropout_0.06,4.130373623222113,-0.018113188445568085
+2,wikitext103_probe_blend,4.111278761178255,static_dropout_0.06,4.130373623222113,-0.019094862043857574
+2,wikitext103_low_decay,4.129110310226679,static_dropout_0.06,4.130373623222113,-0.0012633129954338074
+2,static_dropout_0.1,4.131969079375267,static_dropout_0.06,4.130373623222113,0.0015954561531543732
+2,static_dropout_0.08,4.137437805533409,static_dropout_0.06,4.130373623222113,0.007064182311296463
+2,static_dropout_0.06,4.130373623222113,static_dropout_0.06,4.130373623222113,0.0
+2,static_dropout_0.14,4.147622540593147,static_dropout_0.06,4.130373623222113,0.017248917371034622
+2,static_dropout_0.18,4.147120479494333,static_dropout_0.06,4.130373623222113,0.01674685627222061
+2,static_dropout_0.2,4.163300380110741,static_dropout_0.06,4.130373623222113,0.032926756888628006
+2,static_dropout_0.04,4.164802756160498,static_dropout_0.06,4.130373623222113,0.03442913293838501
+2,static_dropout_0.02,4.174610733985901,static_dropout_0.06,4.130373623222113,0.04423711076378822
+2,static_dropout_0,4.202965669333935,static_dropout_0.06,4.130373623222113,0.07259204611182213
+2,static_dropout_0.26,4.1961489245295525,static_dropout_0.06,4.130373623222113,0.0657753013074398
+2,static_dropout_0.3,4.215538114309311,static_dropout_0.06,4.130373623222113,0.08516449108719826
+3,wikitext103_formula_l12,4.0763493329286575,static_dropout_0.08,4.1035647466778755,-0.027215413749217987
+3,wikitext103_probe_blend,4.093361176550388,static_dropout_0.08,4.1035647466778755,-0.010203570127487183
+3,wikitext103_low_decay,4.100566737353802,static_dropout_0.08,4.1035647466778755,-0.0029980093240737915
+3,static_dropout_0.1,4.111502721905708,static_dropout_0.08,4.1035647466778755,0.007937975227832794
+3,static_dropout_0.08,4.1035647466778755,static_dropout_0.08,4.1035647466778755,0.0
+3,static_dropout_0.06,4.112732540816069,static_dropout_0.08,4.1035647466778755,0.00916779413819313
+3,static_dropout_0.14,4.123979520052671,static_dropout_0.08,4.1035647466778755,0.020414773374795914
+3,static_dropout_0.18,4.128544177860022,static_dropout_0.08,4.1035647466778755,0.024979431182146072
+3,static_dropout_0.2,4.136738710105419,static_dropout_0.08,4.1035647466778755,0.03317396342754364
+3,static_dropout_0.04,4.124624028801918,static_dropout_0.08,4.1035647466778755,0.02105928212404251
+3,static_dropout_0.02,4.144321486353874,static_dropout_0.08,4.1035647466778755,0.04075673967599869
+3,static_dropout_0,4.175786443054676,static_dropout_0.08,4.1035647466778755,0.07222169637680054
+3,static_dropout_0.26,4.179637394845486,static_dropout_0.08,4.1035647466778755,0.07607264816761017
+3,static_dropout_0.3,4.192550577223301,static_dropout_0.08,4.1035647466778755,0.08898583054542542
+4,wikitext103_formula_l12,4.0844879895448685,static_dropout_0.1,4.109558492898941,-0.02507050335407257
+4,wikitext103_probe_blend,4.09542103484273,static_dropout_0.1,4.109558492898941,-0.014137458056211472
+4,wikitext103_low_decay,4.092821758240461,static_dropout_0.1,4.109558492898941,-0.01673673465847969
+4,static_dropout_0.1,4.109558492898941,static_dropout_0.1,4.109558492898941,0.0
+4,static_dropout_0.08,4.122252244502306,static_dropout_0.1,4.109558492898941,0.012693751603364944
+4,static_dropout_0.06,4.118809700012207,static_dropout_0.1,4.109558492898941,0.009251207113265991
+4,static_dropout_0.14,4.111692663282156,static_dropout_0.1,4.109558492898941,0.0021341703832149506
+4,static_dropout_0.18,4.132994774729013,static_dropout_0.1,4.109558492898941,0.023436281830072403
+4,static_dropout_0.2,4.13876885920763,static_dropout_0.1,4.109558492898941,0.029210366308689117
+4,static_dropout_0.04,4.131225533783436,static_dropout_0.1,4.109558492898941,0.02166704088449478
+4,static_dropout_0.02,4.1386831142008305,static_dropout_0.1,4.109558492898941,0.02912462130188942
+4,static_dropout_0,4.185262691229582,static_dropout_0.1,4.109558492898941,0.07570419833064079
+4,static_dropout_0.26,4.178157042711973,static_dropout_0.1,4.109558492898941,0.06859854981303215
+4,static_dropout_0.3,4.200709246098995,static_dropout_0.1,4.109558492898941,0.09115075320005417
+5,wikitext103_formula_l12,4.068591594696045,static_dropout_0.08,4.105628222227097,-0.037036627531051636
+5,wikitext103_probe_blend,4.106558587402105,static_dropout_0.08,4.105628222227097,0.0009303651750087738
+5,wikitext103_low_decay,4.102147191762924,static_dropout_0.08,4.105628222227097,-0.0034810304641723633
+5,static_dropout_0.1,4.118561428040266,static_dropout_0.08,4.105628222227097,0.01293320581316948
+5,static_dropout_0.08,4.105628222227097,static_dropout_0.08,4.105628222227097,0.0
+5,static_dropout_0.06,4.125324193388224,static_dropout_0.08,4.105628222227097,0.01969597116112709
+5,static_dropout_0.14,4.1191585548222065,static_dropout_0.08,4.105628222227097,0.01353033259510994
+5,static_dropout_0.18,4.132549241185188,static_dropout_0.08,4.105628222227097,0.026921018958091736
+5,static_dropout_0.2,4.141841117292643,static_dropout_0.08,4.105628222227097,0.036212895065546036
+5,static_dropout_0.04,4.141878347843885,static_dropout_0.08,4.105628222227097,0.036250125616788864
+5,static_dropout_0.02,4.134641747921705,static_dropout_0.08,4.105628222227097,0.02901352569460869
+5,static_dropout_0,4.193427540361881,static_dropout_0.08,4.105628222227097,0.0877993181347847
+5,static_dropout_0.26,4.18209795281291,static_dropout_0.08,4.105628222227097,0.07646973058581352
+5,static_dropout_0.3,4.184087038040161,static_dropout_0.08,4.105628222227097,0.07845881581306458

runs/wikitext103_streaming_report/l12_validation_5seed/stage_summary.csv

@@ -0,0 +1,71 @@
+condition,stage,token_limit,dropout,n,mean_val,std_val,mean_train,std_train,mean_gap,std_gap
+wikitext103_formula_l12,0,250000,0.3,5,5.21483291387558,0.018060268226475495,4.213146224617958,0.013218263380840874,1.0016866892576217,0.027238437205160136
+wikitext103_formula_l12,1,500000,0.26,5,4.80534844994545,0.027829930328897787,3.918210099637508,0.011987444359375697,0.8871383503079414,0.02626272167699895
+wikitext103_formula_l12,2,1000000,0.18,5,4.49377838075161,0.014674108647494532,3.8283171117305757,0.017535628899982475,0.6654612690210342,0.02707089996550965
+wikitext103_formula_l12,3,2000000,0.09,5,4.260725708305836,0.018098739272622868,3.8088956013321877,0.016376701234484724,0.45183010697364806,0.01960955973506382
+wikitext103_formula_l12,4,4000000,0.02,5,4.080797865241766,0.019458062276034267,3.7991417959332465,0.028273745467366926,0.2816560693085194,0.038849358858741
+wikitext103_probe_blend,0,250000,0.19,5,5.165890334546566,0.017075109484847858,4.020059056580067,0.016213485710936643,1.1458312779664994,0.02016518864116932
+wikitext103_probe_blend,1,500000,0.14,5,4.7872447416186334,0.0268838649061173,3.684571087360382,0.02397149156090502,1.1026736542582511,0.021535214146850536
+wikitext103_probe_blend,2,1000000,0.09,5,4.49399798810482,0.015864095933133645,3.6495467707514764,0.019490464893964263,0.8444512173533439,0.03475725994195922
+wikitext103_probe_blend,3,2000000,0.04,5,4.274319227784872,0.0173783813582558,3.720025636255741,0.012889615922098394,0.5542935915291309,0.012746502538982045
+wikitext103_probe_blend,4,4000000,0.01,5,4.096089626103639,0.014523730760550201,3.7674024373292925,0.027214408565503653,0.3286871887743473,0.038218075738052845
+wikitext103_low_decay,0,250000,0.14,5,5.163547059893608,0.02195330299872749,3.9050611212849615,0.01328398150034886,1.2584859386086464,0.025343680837745006
+wikitext103_low_decay,1,500000,0.14,5,4.800087235867977,0.01984656043518159,3.6649743929505347,0.023120354646606448,1.1351128429174424,0.01298098541252223
+wikitext103_low_decay,2,1000000,0.1,5,4.50133255571127,0.01578619433972172,3.6606912568211554,0.023899445408052746,0.8406412988901139,0.03886870429434243
+wikitext103_low_decay,3,2000000,0.06,5,4.273568108677864,0.022770462958788148,3.7474397838115694,0.003830485352807811,0.5261283248662949,0.02051193157067845
+wikitext103_low_decay,4,4000000,0.02,5,4.102013133466244,0.016554390946653275,3.776866267621517,0.02647416884246062,0.3251468658447266,0.03314100249583806
+static_dropout_0.1,0,250000,0.1,5,5.169893845915794,0.023739040366881205,3.8219083666801454,0.019248328397722874,1.3479854792356492,0.03169483270954973
+static_dropout_0.1,1,500000,0.1,5,4.83315060287714,0.025840187319246186,3.5636927232146265,0.024862277351108786,1.2694578796625138,0.01835928668354016
+static_dropout_0.1,2,1000000,0.1,5,4.518611620366573,0.01351751181040844,3.652407945692539,0.022251919818600654,0.8662036746740341,0.033766642849086155
+static_dropout_0.1,3,2000000,0.1,5,4.285703422129155,0.018994111876547076,3.7808884128928186,0.01766528157534878,0.5048150092363357,0.028072111875115907
+static_dropout_0.1,4,4000000,0.1,5,4.1104619488120075,0.018806640195540816,3.841722105443478,0.019884552697005082,0.2687398433685303,0.03219305563944893
+static_dropout_0.08,0,250000,0.08,5,5.189354091882706,0.016114118388750917,3.761947725713253,0.017361993699706244,1.4274063661694527,0.012218707428925264
+static_dropout_0.08,1,500000,0.08,5,4.857585413753986,0.023903772104159438,3.50356438010931,0.024856947856236954,1.3540210336446763,0.0310369653543579
+static_dropout_0.08,2,1000000,0.08,5,4.532606348395348,0.011746366682868653,3.5999757796525955,0.009078845784580064,0.9326305687427521,0.019403758262432406
+static_dropout_0.08,3,2000000,0.08,5,4.292461846768856,0.016558424281915483,3.7518652200698854,0.019831550751639896,0.5405966266989708,0.034040566494281235
+static_dropout_0.08,4,4000000,0.08,5,4.111632210761309,0.01857117977689519,3.8268220275640488,0.021203534366681782,0.2848101831972599,0.03679859980086126
+static_dropout_0.06,0,250000,0.06,5,5.215402702987194,0.01734079554543704,3.712760145962238,0.018437630763482243,1.5026425570249557,0.03154944432034942
+static_dropout_0.06,1,500000,0.06,5,4.894671627879143,0.021331675814859084,3.4394222095608713,0.013418585935883633,1.4552494183182716,0.01946001354297248
+static_dropout_0.06,2,1000000,0.06,5,4.557375983893872,0.012636266859277006,3.555374290049076,0.018874416206200166,1.0020016938447953,0.030341255269329737
+static_dropout_0.06,3,2000000,0.06,5,4.305900506675243,0.018234543669842573,3.7317687764763834,0.017157479537739013,0.5741317301988602,0.023633182905307998
+static_dropout_0.06,4,4000000,0.06,5,4.119682380557061,0.008179605355252681,3.806561966240406,0.022924307415498075,0.3131204143166542,0.028667268842643943
+static_dropout_0.14,0,250000,0.14,5,5.1635470792651175,0.021953339409384615,3.9050611019134522,0.013283976741307785,1.2584859773516655,0.025343683677925957
+static_dropout_0.14,1,500000,0.14,5,4.800087215006352,0.019846558873074828,3.6649743527173997,0.023120394246437293,1.135112862288952,0.012980966896977213
+static_dropout_0.14,2,1000000,0.14,5,4.500055414438248,0.014819407090300617,3.716250878572464,0.02407671653248301,0.7838045358657837,0.038101035301189086
+static_dropout_0.14,3,2000000,0.14,5,4.281563547253609,0.02138858846966249,3.828673002123833,0.004937581309037719,0.452890545129776,0.01755457638984346
+static_dropout_0.14,4,4000000,0.14,5,4.122133788466454,0.015529540565590345,3.8673695802688597,0.025363134860510252,0.2547642081975937,0.0302072467927019
+static_dropout_0.18,0,250000,0.18,5,5.16162400841713,0.014987630048664583,3.9964362382888794,0.01862825818296394,1.16518777012825,0.02013674124988852
+static_dropout_0.18,1,500000,0.18,5,4.794638857245445,0.02062163611688947,3.7571836963295935,0.024732309568709827,1.0374551609158515,0.02312552092553772
+static_dropout_0.18,2,1000000,0.18,5,4.5023036181926726,0.018490102933264013,3.786629955470562,0.015769997442310384,0.7156736627221107,0.03180040950921459
+static_dropout_0.18,3,2000000,0.18,5,4.288915157318115,0.019532110178580833,3.876796562969685,0.008357602777023191,0.4121185943484306,0.013904819825939841
+static_dropout_0.18,4,4000000,0.18,5,4.130411610752344,0.013005735782293457,3.9014849200844766,0.018144303418082167,0.22892669066786767,0.02966442514236336
+static_dropout_0.2,0,250000,0.2,5,5.170051643252373,0.014071496005973716,4.036272630095482,0.013668609797313208,1.1337790131568908,0.013715020568700555
+static_dropout_0.2,1,500000,0.2,5,4.787301687896251,0.023601504941398347,3.7914333969354628,0.018987144953080883,0.9958682909607888,0.014572669955232898
+static_dropout_0.2,2,1000000,0.2,5,4.506029562652111,0.02043622151331783,3.814779002964497,0.01587114098616348,0.6912505596876144,0.03147408021666257
+static_dropout_0.2,3,2000000,0.2,5,4.294206875562668,0.016019909189912813,3.898232290148735,0.008893270752025355,0.3959745854139328,0.01268490098888804
+static_dropout_0.2,4,4000000,0.2,5,4.1393771633505825,0.016742174044810716,3.9154964342713354,0.020116706552102667,0.22388072907924653,0.033233695598367155
+static_dropout_0.04,0,250000,0.04,5,5.237822580337524,0.018592151453448197,3.6440532103180887,0.031242522590923742,1.593769370019436,0.03528512909650908
+static_dropout_0.04,1,500000,0.04,5,4.957330641150475,0.02503345339488141,3.3515226930379867,0.029537749484816224,1.6058079481124878,0.0202359151612378
+static_dropout_0.04,2,1000000,0.04,5,4.595940679311752,0.014619476464690098,3.502993068099022,0.017291674978225223,1.0929476112127303,0.028916946275823
+static_dropout_0.04,3,2000000,0.04,5,4.3262467920780185,0.016533384756725625,3.703781445324421,0.01970997472996272,0.6224653467535972,0.02577206799431021
+static_dropout_0.04,4,4000000,0.04,5,4.1331265024840835,0.022679096058077455,3.7978432700037956,0.018504253509089505,0.3352832324802876,0.036956450676459744
+static_dropout_0.02,0,250000,0.02,5,5.274957031011581,0.025475383539799955,3.572465108335018,0.027061999609254494,1.7024919226765634,0.03656967535619302
+static_dropout_0.02,1,500000,0.02,5,5.045114178955555,0.016894569246234076,3.261230443418026,0.014168838694048941,1.783883735537529,0.019840865900389654
+static_dropout_0.02,2,1000000,0.02,5,4.655830132961273,0.01585338982667534,3.4323942199349404,0.018124036007212796,1.2234359130263328,0.024146051923035036
+static_dropout_0.02,3,2000000,0.02,5,4.355083760619164,0.026489436313039863,3.6672953754663467,0.010966930251295475,0.6877883851528168,0.026106538952549097
+static_dropout_0.02,4,4000000,0.02,5,4.145870254188776,0.016455755256661365,3.775918734073639,0.03360785862475549,0.3699515201151371,0.0368559849747316
+static_dropout_0,0,250000,0.0,5,5.340271946787834,0.02699193219713777,3.5230478435754775,0.03634617773629946,1.8172241032123566,0.05178790589031804
+static_dropout_0,1,500000,0.0,5,5.174065832793713,0.025168511150111032,3.1505795806646346,0.025699619025332174,2.0234862521290777,0.0236535174506497
+static_dropout_0,2,1000000,0.0,5,4.766069588065148,0.03242686059194632,3.3657630145549775,0.025194332688414405,1.40030657351017,0.052237334934698405
+static_dropout_0,3,2000000,0.0,5,4.417352265119552,0.01876483626309062,3.6409011498093604,0.014124132971310668,0.7764511153101921,0.024211298020746064
+static_dropout_0,4,4000000,0.0,5,4.183486974239349,0.01652781602088682,3.7750062167644503,0.025453964920340872,0.4084807574748993,0.03358628463770181
+static_dropout_0.26,0,250000,0.26,5,5.194025552272796,0.016128282325428966,4.149615630507469,0.024675006962846267,1.0444099217653275,0.022196467420881313
+static_dropout_0.26,1,500000,0.26,5,4.8081169500947,0.021576250073037834,3.905269515514374,0.02376505939443177,0.9028474345803261,0.018700407757224247
+static_dropout_0.26,2,1000000,0.26,5,4.52620010226965,0.010149133513914643,3.9071271896362303,0.02774032700763299,0.619072912633419,0.035195638721522884
+static_dropout_0.26,3,2000000,0.26,5,4.32477528527379,0.017871972287011016,3.9654731526970863,0.018160558354021094,0.359302132576704,0.018728321745075875
+static_dropout_0.26,4,4000000,0.26,5,4.178351600468159,0.01453281035529064,3.9775424867868425,0.021977989750098077,0.20080911368131638,0.03295020359909027
+static_dropout_0.3,0,250000,0.3,5,5.214832927286625,0.018060234410376495,4.213146212697029,0.013218280901902867,1.0016867145895958,0.02723842931339789
+static_dropout_0.3,1,500000,0.3,5,4.82416096329689,0.029627048936626245,3.976513123512268,0.010796527385978102,0.8476478397846222,0.026893359245285242
+static_dropout_0.3,2,1000000,0.3,5,4.546238152682781,0.012672630425565377,3.970817744731903,0.017137273586986676,0.5754204079508781,0.02420747470801454
+static_dropout_0.3,3,2000000,0.3,5,4.34667577445507,0.015854348211857338,4.009651578962803,0.014639843511797184,0.3370241954922676,0.019075525784314876
+static_dropout_0.3,4,4000000,0.3,5,4.194609892368317,0.014131822174274241,4.012722708284855,0.024177126067976527,0.18188718408346177,0.031100561007876955