--- license: mit library_name: stable-baselines3 tags: - reinforcement-learning - board-game - self-play - maskable-ppo - sb3-contrib - ankh language: - en - pt --- # Ankh: Gods of Egypt — MaskablePPO agent (2–5 players) A single neural network that plays the board game **Ankh: Gods of Egypt** with 2 to 5 players, trained with `MaskablePPO` (sb3-contrib) through self-play against earlier snapshots of itself. > **Fan project.** Not affiliated with, endorsed by, or connected to CMON, the game's designers or publishers. > The game rules were re-implemented in Python from the public rulebook for research and learning. This repository > contains **no** rulebook, artwork or other game assets. **Resumo em português:** agente de PPO com máscara de ações que joga Ankh de 2 a 5 jogadores, treinado em self-play (16 milhões de passos, 40 rounds). O repositório inclui o modelo, o ambiente de regras e um servidor local (`jogar.py`) para jogar contra a IA no navegador. Os resultados abaixo são fracos como medida de força absoluta: veja "Limitações". ## Model | | | |---|---| | Algorithm | `MaskablePPO` (sb3-contrib), `MlpPolicy`, net_arch `[512, 512]` | | Checkpoint | `round_0040.zip` (round 40, ≈16.0M timesteps) | | Observation | 6786 floats in [0, 1], always from the point of view of the deciding seat (me + up to 4 opponents, zero-padded) | | Actions | 332 discrete actions, with an action mask for every decision | | Players | one network for 2, 3, 4 and 5 players | | Hidden information | the encoder never reads other players' pending battle cards or Plague bids | | Versions used | stable-baselines3 2.9.2, PyTorch 2.15, Gymnasium 1.4.0 | Hyperparameters: `n_steps=128`, `n_epochs=4`, `gamma=0.995`, `gae_lambda=0.95`, `ent_coef=0.01`, `learning_rate=3e-4`, batch size = `n_steps * n_envs / 4`. ## How to use ```bash pip install -r requirements.txt # plus PyTorch ``` Load the model (the action mask is required): ```python from huggingface_hub import hf_hub_download from sb3_contrib import MaskablePPO path = hf_hub_download("heitorvcm/ankh-ppo", "round_0040.zip") model = MaskablePPO.load(path, device="cpu") # obs: np.ndarray (6786,) from ankh_obs.Encoder(game).encode(decision) # mask: np.ndarray (332,) from decision.mask action, _ = model.predict(obs, action_masks=mask, deterministic=False) ``` The observation encoder and the rules live in this repository (`ankh_obs.py`, `ankh_game.py`); **the checkpoint is useless without them**, and checkpoints from other versions of the encoder will not load. Play against it in your browser (local server, no data leaves your machine): ```bash python jogar.py --models round_0040.zip # open http://127.0.0.1:8765 ``` Print a full text replay of a game between copies of the model (shows hidden information; debugging tool): ```bash python replay.py --scenario basic_4p --seed 7 --models round_0040.zip,round_0040.zip,round_0040.zip,round_0040.zip ``` ## Training setup - **Game:** 4 scenarios (`basic_2p` … `basic_5p`), all 5 gods (Amun, Anubis, Isis, Osiris, Ra), random seat and gods every game, god merging with 3+ players. Scenario data was typed by hand and is not verified against the official scenario book. - **Opponents:** random at the start, then snapshots of the model itself (about 50% the latest snapshot, the rest mostly recent ones with some older ones). - **Reward:** +1 win, −1/(P−1) loss, −1 if every god is forgotten; small dense terms: +0.02 per Devotion step gained, −0.01 × mean opponent steps, +0.005 per warrior added on the board, −0.0025 × mean opponent change. The god merge itself gives and takes no reward (accumulated Devotion is used). - **Compute:** CPU workers for the rules, GPU for inference and updates; 40 rounds × 400k steps, ≈445 minutes. ## Results (read the caveats) All numbers come from the training log of the last round (4,348 games against the round's snapshot opponent; ties to chance are computed against `1/players`). | Players | Games | Agent win rate | Chance (1/P) | |---|---|---|---| | 2 | 1052 | 57% | 50% | | 3 | 1081 | 48% | 33% | | 4 | 1123 | 30% | 25% | | 5 | 1092 | 24% | 20% | Overall 39% vs. roughly 32% expected by chance with this mix of player counts. Against its previous snapshot in the evaluation step (160 games, deterministic agent vs. sampling opponent) it won 54%. **Caveats** - The only opponents were earlier copies of the same model. There is **no** evaluation against human players or an independent baseline, and the evaluation against random opponents was not recorded for this run. - The training win rate has been flat at ≈39–46% since round 29: no clear sign of continued improvement. - Strong **seat effects**: in 2-player games the agent wins 85% as seat 0 and 27% as seat 1; with 4 players seat 0 wins 60% and the others 17–22%; with 5 players seat 0 is the *worst* (12%). The cause is unknown (turn order, event timing or scenario setup). - In two inspected replays the agent rarely used the *Move Figures* action (≈8–12% of actions) and chose the *Unlock Ankh Power* action as a "pass" once all powers were unlocked (allowed by the rules). Level-3/2 powers Glorious, Magnanimous and Obelisk Attuned were never unlocked by the agent or its opponents. - Being in a god merge happened in 40% of 3+ player games (about 52% expected for a random seat), so there is no sign that the agent seeks the merge. This is a statistic over one run, not a guarantee. - Several rules interpretations are assumptions (see `README_projeto_pt.md`); the model is only as faithful as that implementation. ## Intended use Research, education and casual play against a bot. Not a strength benchmark for the real board game. ## Security note Stable-Baselines3 checkpoints are zip files that use Python pickle. Only load checkpoints from sources you trust. ## Files | File | What it is | |---|---| | `round_0040.zip` | the trained model | | `ankh_game.py`, `ankh_map.py` | rules and board/scenario data | | `ankh_obs.py` | observation encoder | | `ankh_env.py`, `ankh_worker.py`, `ankh_vec.py`, `ankh_eval.py`, `ankh_report.py`, `train_local.py`, `report.py` | training and evaluation code | | `replay.py` | text replay of games | | `jogar.py`, `jogar.html` | local web game against the AI | | `README_projeto_pt.md` | original project notes (Portuguese) | | `LICENSE` | license of the code | ## Licence Code and model: mit. The game itself, its name and artwork belong to their respective owners.