Update logbook: Repro - DropoutTS: Sample-Adaptive Dropout for Robust Time Series Forecasting

logbook.json

@@ -10,7 +10,7 @@
     "icml2026-repro",
     "paper-7sksHLUvhH"
   ],
-  "updated_at": "2026-07-16T04:45:03+00:00",
+  "updated_at": "2026-07-16T04:58:15+00:00",
   "root": {
     "slug": "index",
     "title": "Repro - DropoutTS: Sample-Adaptive Dropout for Robust Time Series Forecasting",
@@ -54,6 +54,6 @@
       }
     ]
   },
-  "agent_view_tokens": 3386,
-  "revision": "1784177103295512000"
+  "agent_view_tokens": 3806,
+  "revision": "1784177895091670000"
 }

pages/claim-1-synth-benchmark-informer-improvement/page.md

@@ -3931,3 +3931,10 @@ noise,horizon,mse_base,mse_drop,mse_imp_pct,mae_base,mae_drop,mae_imp_pct,base_s
 0.9,720,0.8700354445091655,0.5207760263559276,40.143125243624326,0.7190576933784872,0.586870783087967,18.383352477523882,62.97391304347826,47.833333333333336,23,30,1448.4,1435.0
 
 ````
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_a789c7016367", "created_at": "2026-07-16T04:58:12+00:00", "title": "Methodology limitation (added after self-review). The results above use a singl…"}
+-->
+Methodology limitation (added after self-review). The results above use a single fixed init_sensitivity=5.0, while the repo's run_baselines.py sweeps init_sensitivity over {1, 5, 10} and would report the best per configuration. So this compares an untuned DropoutTS against the baseline, which is not the paper's protocol. A targeted sensitivity sweep at sigma=0.3 is running now to check whether tuning recovers the claimed improvement; I will update the verdict here with those results. Treat the 'not reproduced' reading as provisional and specifically about the default configuration, not a claim that the method cannot work.

pages/claim-5-orthogonal-compatibility-with-selective-learning/page.md

@@ -8,3 +8,10 @@
 Claim 5 says combining DropoutTS with a Selective Learning strategy beats Selective Learning alone (21.2 vs 19.6 percent), so the two are meant to be orthogonal and stack. I tested this on the synthetic sigma=0.3 set with Informer across three conditions: a plain baseline, Selective Learning alone (uncertainty masking, r_u=0.1), and DropoutTS plus Selective Learning. Improvements over baseline came out at +23.3 percent for Selective Learning alone and +19.2 percent for the combination. The combination was worse than Selective Learning by itself, so the orthogonal stacking the claim describes did not show up. Here DropoutTS took away from the data-centric strategy rather than adding to it.
 
 Caveats: one dataset, one seed, and I chose r_u=0.1 because the claim does not pin down the Selective Learning configuration. I also had to patch a bug to run this at all. The repo's SelectiveLearning callback calls to_device on a None estimator in uncertainty-only mode and crashes with 'NoneType object has no attribute cuda'. I added a guard so the r_u-only path runs. That crash is itself a small reproducibility finding. Run on Modal: https://modal.com/apps/lexigenai/main/ap-QlN4lF2pNbSgtVOjB0h8tL
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_4edacc50c92f", "created_at": "2026-07-16T04:58:12+00:00", "title": "Two limitations to note (added after self-review). First, single seed and a sin…"}
+-->
+Two limitations to note (added after self-review). First, single seed and a single r_u=0.1, so this is one operating point, not a tuned comparison. Second, an earlier version of this run used use_clean_targets=False, which is inconsistent with the repo's USE_CLEAN_TARGETS=True convention and with the Claim 1 setup; the numbers above are being re-run with use_clean_targets=True for consistency, and I will update them. Treat the 'no orthogonal gain' reading as provisional.