Cochon123/tileman-io-ai

Tileman AI Lab

A local Tileman.io-style multiplayer simulator, live AI spectator, and compute-efficient reinforcement-learning environment. The simulation core is written in Rust, the training interface uses Python/Gymnasium, and the spectator runs in the browser.

Quick start

Requirements

• Linux or macOS
• Python 3.11 or newer
• Node.js 18 or newer
• Rust toolchain (only required for the fast native simulator)
• Optional: an NVIDIA GPU with a working CUDA build of PyTorch

Clone the code and create an isolated Python environment:

git clone https://github.com/cochon123/tileman-io-ai.git
cd tileman-io-ai

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements-rl.txt
python -m pip install maturin huggingface_hub

Build the optimized Rust simulator:

maturin develop --release

Download the three published best models from Hugging Face:

hf download Cochon123/tileman-io-ai --local-dir runs

Start the world builder:

npm run dev

Open http://127.0.0.1:4173. Choose a world size, simulation speed, AI policies, and the number of instances of each policy, then press Start.

The selector includes the three main policies:

• first: the original conservative capture policy
• champion: the tournament-selected multiplayer champion
• killer: the fresh-from-scratch combat policy

The world builder lets you choose:

• A preset or custom world size and simulation speed
• One to three trained AI policies
• The exact number of instances controlled by each policy, up to 48 agents

Starting a simulation loads the selected policies on CUDA and opens the live world. Use the Exit button in the spectator header to stop that simulation and return to the world builder.

If CUDA is unavailable, change the device to cpu in direct Python commands. You can verify GPU support with:

python -c "import torch; print(torch.cuda.is_available())"

Installation troubleshooting

• If maturin develop cannot find Rust, install it from https://rustup.rs, restart the shell, and rerun the command.
• If the Rust extension is not built, the environments still run using the slower reference Python implementation.
• If hf is not found, reactivate .venv or run python -m pip install huggingface_hub.
• The first CUDA launch may take a few seconds while PyTorch initializes and loads the policies.

Trained multi-agent spectator

Run 32 clones of the best trained PPO policy on one shared 256×180 world:

npm run spectate -- --device cuda

Open http://127.0.0.1:4174. The first launch can take a little while while PyTorch initializes CUDA and loads the model.

Spectator controls:

• Click any name in the left leaderboard to smoothly follow that AI.
• Scroll anywhere over the world to zoom without releasing the followed AI.
• Use WASD, the arrow keys, or pointer dragging to release follow mode and explore freely.
• Use Fit world to detach the camera and frame the complete map.
• Click another leaderboard name at any time to resume smooth follow mode.

The server batches every visible clone through runs/ppo-conservative/best/best_model.zip; these are instances of the trained policy, not scripted browser bots. World size, population, model, and tick rate are configurable:

python3 -m tileman_rl.spectator_server \
  --model runs/ppo-conservative/best/best_model.zip \
  --agents 32 \
  --cols 256 \
  --rows 180 \
  --tick-rate 12 \
  --device cuda

To watch the Killer policy, select its checkpoint explicitly:

npm run spectate -- \
  --model runs/ppo-killer-v1-cuda/best/best_model.zip \
  --device cuda

The spectator detects whether a model uses the original four observation channels or the multiplayer seven-channel view, so both generations of checkpoints remain usable.

To place half of the old agents and half of the new agents on the same map:

npm run spectate -- \
  --model runs/ppo-conservative/best/best_model.zip \
  --model-b runs/ppo-champion/best_model.zip \
  --agents 32 \
  --device cuda

The leaderboard labels the two groups OLD and NEW. With an odd population, the extra agent is assigned to the new model.

The same champion match has a shortcut:

npm run spectate:champion

To compare all three generations with 11 agents each:

npm run spectate:all

This runs the original model (OLD), the fresh-from-scratch combat policy (KILLER), and the tournament-selected champion (CHAMP) on one shared map. Policy assignments are interleaved by agent ID to reduce movement-order bias.

Current champion

runs/ppo-champion/best_model.zip was selected by paired-seed tournaments, not training reward. Every world seed is replayed with model assignments swapped to cancel spawn and agent-ID advantages. In the final 32-agent, 256×180, 1,000-tick validation, the champion controlled first place for 56.4% of ticks versus 43.7% for the old model, with more captures and kills and slightly fewer deaths.

The champion is an evolved variant of the old capture policy with a small deterministic action-logit adjustment, discovered after PPO league branches were rejected whenever they failed the same tournament. Reproduce a comparison with:

python3 -m tileman_rl.tournament \
  runs/ppo-champion/best_model.zip \
  --baseline runs/ppo-conservative/best/best_model.zip \
  --matches 4 --agents 32 --cols 256 --rows 180 \
  --ticks 1000 --device cuda

Run the JavaScript model tests with:

npm test

AI interface

The browser exposes the environment at window.tileman:

tileman.reset();
const transition = tileman.step("right");
console.log(transition.observation, transition.reward, transition.terminated);

Actions are up, right, down, and left. An observation contains a flat Uint8Array grid (0 empty, 1 owned, 2 trail), grid shape, player position/direction, trail length, area, and step count.

The browser reward now mirrors the conservative training reward: captures are positive, dying is -1, and exposed trails longer than 12 cells become progressively more expensive.

Reinforcement-learning environment

The Python environment uses:

• A configurable 128×128 world by default
• A rotated 43×25 semantic viewport centred on the player
• Three relative actions: straight, left, and right
• Compact statistics: trail length, time outside, direction to home, and coverage
• No browser or rendering work during training

The reward is:

+0.01 × newly captured tiles
-0.0001 per step
-0.002 per step outside owned territory
-0.001 × every trail cell beyond 12
-1.0 on death

The excess-trail cost is applied on every exposed step. Long trails therefore become increasingly unattractive, but remain possible when a large capture justifies the risk.

Run all Python contract and rule tests:

python3 -m unittest discover -s tests -v

Rust simulator

The single-agent and multiplayer training environments automatically use a native Rust core when it is installed. The multiplayer training core keeps the world, scripted opponent plans, trail kills, capture flood fill, respawns, rewards, and observation buffers in Rust, crossing into Python once per Gym step. The general shared-world class also has a compatibility-oriented Rust backend while preserving its mutable NumPy grids for spectators and policy leagues.

Build the extension in a virtual environment:

uv venv
uv pip install -r requirements-rl.txt
uv pip install maturin
uv run maturin develop --release

Confirm the selected backend and compare it with the NumPy reference:

python3 -c "from tileman_rl.env import TilemanTrainingEnv; print(TilemanTrainingEnv.backend)"
python3 -m benchmarks.benchmark_env --steps 50000
python3 -m benchmarks.benchmark_multiplayer --steps 30000

If the native module has not been built, both environments safely fall back to their reference Python implementations. Pass backend="python" to MultiAgentWorld or MultiplayerTilemanEnv for differential debugging; PythonTilemanTrainingEnv remains available for single-agent parity testing.

Start a short CPU experiment:

python3 -m tileman_rl.train \
  --timesteps 100000 \
  --n-envs 8 \
  --device cpu \
  --output runs/ppo-conservative

Training begins with 20,000 actions from the conservative rectangle expert and four balanced behavior-cloning epochs. This prevents PPO from discovering the safe-but-useless strategy of circling inside its spawn forever. The expert bootstrap can be changed or disabled:

# Faster development probe
python3 -m tileman_rl.train --timesteps 10000 --expert-steps 5000 --bc-epochs 3

# PPO from scratch, mainly for comparison
python3 -m tileman_rl.train --timesteps 100000 --expert-steps 0

Once python3 -c 'import torch; print(torch.cuda.is_available())' returns True, use the GPU:

python3 -m tileman_rl.train \
  --timesteps 1000000 \
  --n-envs 8 \
  --device cuda \
  --output runs/ppo-conservative

Evaluate a checkpoint over fixed deterministic seeds:

python3 -m tileman_rl.evaluate \
  runs/ppo-conservative/best/best_model.zip \
  --episodes 100

Evaluation reports area, captures, survival, peak trail length, reward, and death causes. TensorBoard is optional; install it separately if training charts are wanted.

Multiplayer fine-tuning

The multiplayer environment puts one learning agent and six opponents in a shared world. Its seven-channel local observation adds enemy territory, trails, and heads to the original four channels. Existing capture models are widened from four to seven channels while preserving all learned weights.

Training uses two stages:

1. A short combat curriculum exposes vulnerable enemy trails and gives a temporary small kill reward so combat is no longer an extremely rare learning event.
2. Outcome training removes the kill reward and rewards survival, final rank, and held territory while some opponents actively hunt exposed trails.

Run the complete curriculum from the original capture model:

python3 -m tileman_rl.train_multiplayer \
  --source runs/ppo-conservative/best/best_model.zip \
  --timesteps 1000000 \
  --combat-fraction 0.3 \
  --n-envs 8 \
  --device cuda \
  --output runs/ppo-multiplayer

Continue outcome-only training from an already widened combat model:

python3 -m tileman_rl.train_multiplayer \
  --source runs/ppo-multiplayer-v2/combat_model.zip \
  --timesteps 700000 \
  --combat-fraction 0 \
  --n-envs 8 \
  --device cuda \
  --output runs/ppo-multiplayer-final

Evaluate under normal or trail-hunting opponents:

python3 -m tileman_rl.evaluate_multiplayer \
  runs/ppo-multiplayer-final/best/best_model.zip \
  --episodes 100 --envs 8 --device cuda

python3 -m tileman_rl.evaluate_multiplayer \
  runs/ppo-multiplayer-final/best/best_model.zip \
  --episodes 100 --envs 8 --device cuda \
  --aggressive-opponents

Scope of this first version

• Grid movement and four discrete actions
• Vulnerable trails outside owned territory
• Flood-fill territory capture after reconnecting
• Boundary and self-trail terminal conditions
• Deterministic baseline bot and episode metrics
• Headless Gymnasium environment with local partial observations
• Compact custom CNN and PPO training pipeline
• Balanced behavior cloning from a conservative short-loop expert
• Deterministic checkpoint evaluation
• Shared-world multiplayer collision, trail-kill, rank, and respawn rules
• Seven-channel enemy-aware observations and staged multiplayer PPO fine-tuning
• Paired-seed mixed-policy tournament selection and evolved champion checkpoints