| license: mit | |
| tags: | |
| - reinforcement-learning | |
| - ppo | |
| - openfront | |
| - game-ai | |
| # OpenFront RL Agent | |
| PPO-trained agent for [OpenFront.io](https://openfront.io), a multiplayer territory control game. | |
| ## Training Details | |
| - **Algorithm:** PPO (Proximal Policy Optimization) | |
| - **Architecture:** Actor-Critic with shared backbone (512→512→256) | |
| - **Observation dim:** 80 | |
| - **Max neighbors:** 16 | |
| - **Maps:** plains, big_plains, world, giantworldmap, ocean_and_land, half_land_half_ocean, europe, europeclassic, northamerica, africa, asia, australia, southamerica, mediterranean, britannia, britanniaclassic, eastasia, oceania, pangaea, mena, aegean, alps, amazonriver, amazonriverwide, arctic, baikal, beringstrait, betweentwoseas, blacksea, bosphorusstraits, deglaciatedantarctica, falklandislands, faroeislands, fourislands, gatewaytotheatlantic, gulfofstlawrence, halkidiki, hawaii, iceland, italia, japan, lemnos, lisbon, manicouagan, niledelta, passage, sanfrancisco, straitofgibraltar, straitofhormuz, surrounded, thebox, theboxplus, tourney1, tourney2, tourney3, tourney4, tradersdream, twolakes, worldrotated, yenisei, achiran, mars, milkyway, montreal, newyorkcity, pluto, reglaciatedantarctica (random per episode) | |
| - **Opponents:** 2 Easy bots | |
| - **Parallel envs:** 8 | |
| - **Learning rate:** 0.00015 | |
| - **Rollout steps:** 512 | |
| - **Updates trained:** 2750 | |
| - **Global steps:** 11264000 | |
| - **Best mean reward:** 2761.6309762120245 | |
| ## Final Training Metrics | |
| - **Mean reward:** 1866.676198823452 | |
| - **Mean episode length:** 35228.65 | |
| - **Loss:** 17.78866958618164 | |
| ## Usage | |
| ```python | |
| from train import ActorCritic | |
| import torch | |
| model = ActorCritic(obs_dim=80, max_neighbors=16, hidden_sizes=[512, 512, 256]) | |
| model.load_state_dict(torch.load("best_model.pt", weights_only=True)) | |
| model.eval() | |
| ``` | |
| ## Repository | |
| Trained from [josh-freeman/openfront-rl](https://github.com/josh-freeman/openfront-rl). | |