| # Experiment Protocol | |
| ## Runs | |
| - Run ID format: | |
| YYYYMMDD-HHMMSS_<tag>_seed<seed> | |
| ## Training | |
| - Train structured S and dense D_a side-by-side on the same batch stream. | |
| - Update order per step: | |
| 1) forward/backward/step S | |
| 2) forward/backward/step D_a | |
| - Eval cadence: every 500–1000 steps (fixed). | |
| ## Splits | |
| - Fixed IID eval set | |
| - Fixed OOD eval set | |
| - Fixed structure-break eval set (e.g., bind-shuffle) | |
| ## Metrics | |
| Primary: | |
| - IID accuracy/loss | |
| - OOD accuracy/loss | |
| - structure-break accuracy/loss | |
| Secondary (structured-only): | |
| - expert usage entropy | |
| - hemisphere disagreement vs error (simple AUROC) | |
| ## Early-kill criteria | |
| PoC success if ANY: | |
| 1) S > D_a on OOD by clear margin at same budget. | |
| 2) S degrades less under structure-break perturbation. | |
| 3) Disagreement is a useful error signal. | |
| PoC failure: | |
| - No meaningful separation on OOD/robustness and no disagreement signal within a small budget. | |
| ## Artifacts | |
| All outputs must go under: | |
| /store/speedy/dualbrain/out/bihmoe-poc/<run_id>/ | |