racingmania-size-1-seed1

Trained on 1 track, seed 1: PPO trained on only 1 generated tracks

Status: evaluated (evaluated; scores available); physics version v1; card generated 2026-11-15 from the result files.

RacingMania is an original 3D time-attack racing game for the browser (three.js + cannon-es RaycastVehicle) with reinforcement-learning drivers trained on a batched PyTorch port of the same car physics. Code, paper and full results: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Videos

Official evaluation runs of this model replayed in the browser game ("Watch replay"): the car states the evaluator recorded while scoring the run, so each clip is exactly the run its caption describes. Clip numbers are those of the README, which has more clips, including the controls: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game#videos.

Best lap of the single-track ablation

  • What you see: The training-set-size ablation with one track: the standard recipe trained on a single generated track from the pool. The hand-made track was never part of its training; the results compare how far one training track generalizes against the full pool (hypothesis H2).
  • Model: size-1/seed1: PPO, trained by reinforcement learning for 1 billion agent steps on 1 track (seed 1, 1.3 h on one GPU).
  • Map: First Flight (campaign-01), a hand-made campaign track, 628 m long with 1 jump and up to 6° of banking; this model never saw it during training.
  • Evaluation: official evaluation in results/size-1/seed1/eval_campaign.csv (standing start, most probable action, final checkpoint): finished in 20.14 s; the scripted driver needs 30.17 s, so this is 1.50× its speed. The clip replays this run.
  • Why this clip: chosen by a fixed rule as the highest-scoring finished track of the 2 campaign tracks it finished in its official evaluation.

Model

Item Value
Algorithm PPO
Architecture MLP 2x256 tanh (actor and critic), two 3-way heads
Objective centreline progress
Observation car state + 20 look-ahead points
Training data 1 generated tracks
Start states standing + random centreline (p=0.5)
Agent steps trained 1,000,341,504
Wall-clock hours 1.33
Git commit 3e75ae37db322a34f61d1adaf7435d76f72c821b

Evaluation

Success-weighted speed (SWS): mean over tracks of finished x (scripted driver's time / agent's time); 1.0 means finishing every track as fast as the scripted reference driver. One deterministic run per track from a standing start with the most probable action; intervals are 95 % percentile bootstrap intervals over tracks (one seed).

Track set Tracks SWS [95 % CI] Finished
Campaign (hand-made) 5 0.584 [0.000, 1.184] 40.0 %
Out-of-range test 187 0.214 [0.142, 0.291] 14.4 %
Test (unseen generated) 971 0.539 [0.495, 0.583] 36.5 %
Unfiltered test 200 0.544 [0.445, 0.644] 36.0 %
Own training tracks 1 1.636 [1.636, 1.636] 100.0 %
Training sample 1000 0.499 [0.455, 0.542] 33.5 %
Validation 197 0.508 [0.408, 0.607] 34.0 %

How to use

Files: final.pt (PyTorch checkpoint: agent and norm state dicts, ppo and env_cfg settings, obs_dim), policy.onnx (observation function + frozen normaliser + actor in one graph, as run by the browser game), run.json, training and validation curves, and per-track evaluation CSVs under eval/.

PyTorch, inside a clone of the repository (uv sync):

import torch
from racingmania.env.obs import obs_size
from racingmania.rl.networks import ActorCritic
from racingmania.rl.normalizer import RunningNorm

sd = torch.load("final.pt", map_location="cpu", weights_only=False)
k = sd["env_cfg"]["window_k"]
agent = ActorCritic(obs_size(k), tuple(sd["ppo"]["hidden"]), sd["ppo"]["action_mode"])
agent.load_state_dict(sd["agent"]); agent.eval()
norm = RunningNorm(obs_size(k)); norm.load_state_dict(sd["norm"]); norm.frozen = True
# action = agent.act(norm(env.observe().float()), deterministic=True)[1]   (see racingmania.runs.load_policy)

ONNX (inputs and outputs are specified in docs/formats.md, section "Policy model"):

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("policy.onnx")
inputs = {i.name: np.zeros([d if isinstance(d, int) else 1 for d in i.shape], np.float32) for i in sess.get_inputs()}
steer_logits, pedal_logits = sess.run(None, inputs)
steer, pedal = int(steer_logits.argmax()), int(pedal_logits.argmax())  # 0/1/2 = left/none/right, gas/coast/brake

To watch it drive, open the game (web/index.html, or the GitHub Pages site), choose it in a track's AI drivers table and press "Watch AI" (live in the browser) or "Watch replay" (its official evaluation run).

Training configuration (from run.json)

Setting Value
total_steps 1000000000
num_envs 16384
num_steps 32
lr 0.001
adaptive_lr True
desired_kl 0.01
lr_min 1e-05
lr_max 0.01
gamma 0.99
gae_lambda 0.95
num_minibatches 4
update_epochs 5
clip_coef 0.2
ent_coef 0.01
vf_coef 1.0
max_grad_norm 1.0
weight_decay 0.0
hidden [256, 256]
action_mode factored
eval_every_minutes 10.0
checkpoint_every_minutes 10.0
rotate_tracks_every 20
loaded_tracks 4096
seed 1
compile True
env.random_start_prob 0.5
env.random_start_speed [0.0, 30.0]
env.random_start_lateral 0.5
env.max_steps 1500
env.reward_scale 0.1
env.window_k 20
env.window_stride 3
env.blind False
env.time_cost 0.0
env.fall_penalty 0.0
env.hint_weight 0.0
env.scripted_starts False

Licence and citation

MIT. Author: Libor "Burny" Burian. Repository: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Collection including BurnyCoder/racingmania-size-1-seed1

Evaluation results

  • SWS on RacingMania test (unseen generated) tracks
    self-reported
    0.539
  • Finish rate on RacingMania test (unseen generated) tracks
    self-reported
    0.365
  • SWS on RacingMania validation tracks
    self-reported
    0.508
  • Finish rate on RacingMania validation tracks
    self-reported
    0.340
  • SWS on RacingMania training sample tracks
    self-reported
    0.499
  • Finish rate on RacingMania training sample tracks
    self-reported
    0.335
  • SWS on RacingMania own training tracks
    self-reported
    1.636
  • Finish rate on RacingMania own training tracks
    self-reported
    1.000