| # exp190 — saturating soft data without starving training |
|
|
| ## Problem |
| exp186 soft cache was ~56% opening / 22% mid / 22% end. Openings label |
| faster (shallower trees) so naive HF streaming floods the dataset. Training |
| then overfits early-game priors and underperforms in middlegame/endgame — |
| exactly where Elo is decided without MCTS. |
|
|
| ## Data architecture |
| | Lever | Design | |
| |---|---| |
| | Phase quotas | 22% opening / 48% middlegame / 30% endgame (hard gate at writer) | |
| | Depth-by-phase | op 10–14, mid 12–16, eg 14–18 (full-strength SF18, no Elo limit) | |
| | Multi-source FENs | HF stream + book playouts + endgame templates + random walks | |
| | Deficit-first feed | Producer oversamples the lagging phase every cycle | |
| | Syzygy | Wired into each worker when `syzygy/*.rtbw` present | |
| | soft_cache tags | `phase` + `label_depth` tensors for stratified training | |
|
|
| ## Training efficiency (how to consume this) |
| 1. **Stratified batches** — sample 1/3 from each phase bucket every step so |
| gradients never bog on openings. |
| 2. **Depth weights** — `w = label_depth / mean_depth` (or clip 0.5–1.5) so |
| deeper MultiPV pulls harder than shallow noise. |
| 3. **Critical upweight** — boost high `cp_gap_top1_top2` / low entropy rows |
| (forcing moves teach more than quiet equals). |
| 4. **Merge, don't replace** — mix exp190 deep cache with exp186 2M shallow |
| at ~30/70 so you keep volume + add quality. |
| 5. **Round-robin curriculum** — alternate soft-deep / soft-shallow / hard-HF |
| micro-batches inside the same step (already partially done via soft_frac). |
| |
| ## Model-side advantages (no MCTS) |
| - Phase-balanced policy → fewer “opening genius / endgame idiot” failures |
| - Deeper soft targets → better next-move calibration under legal-mask argmax |
| - Endgame + Syzygy teacher → converts tablebase-perfect moves into policy mass |
| - Compact vocab + spatial head already amortizes move representation |
| |
| ## Ops |
| - SF binary: `stockfish/stockfish-latest` → SF18 `x86-64-vnni512` (Xeon 6342) |
| - 64 workers × 96MB hash ≈ 6GB; ~441GB RAM free on this box |
| - Safe alongside GPU exp189 (CPU-only) |
| - Tail: `tail -f outputs/exp190_phase_deep/run.log` |
| |
