File size: 1,287 Bytes
c80bb82 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
license: apache-2.0
tags:
- reinforcement-learning
- self-play
- monte-carlo-tree-search
- policy-value-network
- gradio
---
# MicroZero
MicroZero is a tiny AlphaZero-style Tic-Tac-Toe agent. A shared neural trunk predicts
both move priors and position value. Monte Carlo Tree Search converts those estimates
into stronger self-play targets, while terminal outcomes supervise value learning.
The training loop uses:
- neural-guided PUCT search;
- root Dirichlet exploration;
- policy targets from MCTS visit counts;
- terminal win/draw/loss value targets;
- all eight square-board symmetries;
- a bounded replay buffer.
Evaluation alternates playing first and second against random and exact minimax
opponents.
## Verified results
- 7,626 trainable parameters;
- 980 neural-MCTS self-play games;
- 24,000-example bounded replay buffer after symmetry augmentation;
- 384 wins, 16 draws, and zero losses over 400 games against random play;
- 200 draws and zero losses over 200 games against exact minimax.
The exact opponent is used only for final evaluation, not as a source of training
targets. Full metrics and the per-iteration learning history are stored in
`artifacts/microzero/evaluation.json`.
## Reproduce
```powershell
uv run python projects/microzero/train.py
```
|