| --- |
| license: apache-2.0 |
| tags: |
| - reinforcement-learning |
| - self-play |
| - monte-carlo-tree-search |
| - policy-value-network |
| - gradio |
| --- |
| |
| # MicroZero |
|
|
| MicroZero is a tiny AlphaZero-style Tic-Tac-Toe agent. A shared neural trunk predicts |
| both move priors and position value. Monte Carlo Tree Search converts those estimates |
| into stronger self-play targets, while terminal outcomes supervise value learning. |
|
|
| The training loop uses: |
|
|
| - neural-guided PUCT search; |
| - root Dirichlet exploration; |
| - policy targets from MCTS visit counts; |
| - terminal win/draw/loss value targets; |
| - all eight square-board symmetries; |
| - a bounded replay buffer. |
|
|
| Evaluation alternates playing first and second against random and exact minimax |
| opponents. |
|
|
| ## Verified results |
|
|
| - 7,626 trainable parameters; |
| - 980 neural-MCTS self-play games; |
| - 24,000-example bounded replay buffer after symmetry augmentation; |
| - 384 wins, 16 draws, and zero losses over 400 games against random play; |
| - 200 draws and zero losses over 200 games against exact minimax. |
|
|
| The exact opponent is used only for final evaluation, not as a source of training |
| targets. Full metrics and the per-iteration learning history are stored in |
| `artifacts/microzero/evaluation.json`. |
|
|
| ## Reproduce |
|
|
| ```powershell |
| uv run python projects/microzero/train.py |
| ``` |
|
|
