racingmania-generalist-seed1

Generalist (PPO), seed 1: Generalist PPO agent trained on the whole generated training pool

Status: evaluated (evaluated; scores available); physics version v1; card generated 2026-11-15 from the result files.

RacingMania is an original 3D time-attack racing game for the browser (three.js + cannon-es RaycastVehicle) with reinforcement-learning drivers trained on a batched PyTorch port of the same car physics. Code, paper and full results: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Videos

Official evaluation runs of this model replayed in the browser game ("Watch replay"): the car states the evaluator recorded while scoring the run, so each clip is exactly the run its caption describes. Clip numbers are those of the README, which has more clips, including the controls: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game#videos.

Best lap of the second generalist seed

  • What you see: The generalist of seed 1, trained by reinforcement learning on generated tracks only, as in the preregistered recipe. This hand-made track was never part of its training, so the lap is generalization: it reads the road ahead and picks a speed for each bend.
  • Model: generalist/seed1: PPO, trained by reinforcement learning for 1 billion agent steps on 48,949 generated training tracks (seed 1, 1.9 h on one GPU).
  • Map: Technical (campaign-04), a hand-made campaign track, 666 m long with no jumps and up to 5° of banking; this model never saw it during training.
  • Evaluation: official evaluation in results/generalist/seed1/eval_campaign.csv (standing start, most probable action, final checkpoint): finished in 21.60 s; the scripted driver needs 42.52 s, so this is 1.97× its speed. The clip replays this run.
  • Why this clip: chosen by a fixed rule as the highest-scoring finished track of the 5 campaign tracks it finished in its official evaluation.

Model

Item Value
Algorithm PPO
Architecture MLP 2x256 tanh (actor and critic), two 3-way heads
Objective centreline progress
Observation car state + 20 look-ahead points
Training data 48,949 generated tracks
Start states standing + random centreline (p=0.5)
Agent steps trained 1,000,341,504
Wall-clock hours 1.92
Git commit a861208dbcd896c0f7fe896657c5ee2ca2f45920

Evaluation

Success-weighted speed (SWS): mean over tracks of finished x (scripted driver's time / agent's time); 1.0 means finishing every track as fast as the scripted reference driver. One deterministic run per track from a standing start with the most probable action; intervals are 95 % percentile bootstrap intervals over tracks (one seed).

Track set Tracks SWS [95 % CI] Finished
Campaign (hand-made) 5 1.590 [1.451, 1.783] 100.0 %
Out-of-range test 187 0.875 [0.769, 0.982] 57.8 %
Test (unseen generated) 971 1.214 [1.177, 1.250] 81.5 %
Unfiltered test 200 1.231 [1.147, 1.311] 82.0 %
Own training tracks 1000 1.250 [1.215, 1.284] 83.9 %
Training sample 1000 1.250 [1.215, 1.284] 83.9 %
Validation 197 1.236 [1.153, 1.315] 82.7 %

How to use

Files: final.pt (PyTorch checkpoint: agent and norm state dicts, ppo and env_cfg settings, obs_dim), policy.onnx (observation function + frozen normaliser + actor in one graph, as run by the browser game), run.json, training and validation curves, and per-track evaluation CSVs under eval/.

PyTorch, inside a clone of the repository (uv sync):

import torch
from racingmania.env.obs import obs_size
from racingmania.rl.networks import ActorCritic
from racingmania.rl.normalizer import RunningNorm

sd = torch.load("final.pt", map_location="cpu", weights_only=False)
k = sd["env_cfg"]["window_k"]
agent = ActorCritic(obs_size(k), tuple(sd["ppo"]["hidden"]), sd["ppo"]["action_mode"])
agent.load_state_dict(sd["agent"]); agent.eval()
norm = RunningNorm(obs_size(k)); norm.load_state_dict(sd["norm"]); norm.frozen = True
# action = agent.act(norm(env.observe().float()), deterministic=True)[1]   (see racingmania.runs.load_policy)

ONNX (inputs and outputs are specified in docs/formats.md, section "Policy model"):

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("policy.onnx")
inputs = {i.name: np.zeros([d if isinstance(d, int) else 1 for d in i.shape], np.float32) for i in sess.get_inputs()}
steer_logits, pedal_logits = sess.run(None, inputs)
steer, pedal = int(steer_logits.argmax()), int(pedal_logits.argmax())  # 0/1/2 = left/none/right, gas/coast/brake

To watch it drive, open the game (web/index.html, or the GitHub Pages site), choose it in a track's AI drivers table and press "Watch AI" (live in the browser) or "Watch replay" (its official evaluation run).

Training configuration (from run.json)

Setting Value
total_steps 1000000000
num_envs 16384
num_steps 32
lr 0.001
adaptive_lr True
desired_kl 0.01
lr_min 1e-05
lr_max 0.01
gamma 0.99
gae_lambda 0.95
num_minibatches 4
update_epochs 5
clip_coef 0.2
ent_coef 0.01
vf_coef 1.0
max_grad_norm 1.0
weight_decay 0.0
hidden [256, 256]
action_mode factored
eval_every_minutes 10.0
checkpoint_every_minutes 10.0
rotate_tracks_every 20
loaded_tracks 4096
seed 1
compile True
env.random_start_prob 0.5
env.random_start_speed [0.0, 30.0]
env.random_start_lateral 0.5
env.max_steps 1500
env.reward_scale 0.1
env.window_k 20
env.window_stride 3
env.blind False
env.time_cost 0.0
env.fall_penalty 0.0
env.hint_weight 0.0
env.scripted_starts False

Licence and citation

MIT. Author: Libor "Burny" Burian. Repository: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Collection including BurnyCoder/racingmania-generalist-seed1

Evaluation results

  • SWS on RacingMania test (unseen generated) tracks
    self-reported
    1.214
  • Finish rate on RacingMania test (unseen generated) tracks
    self-reported
    0.815
  • SWS on RacingMania validation tracks
    self-reported
    1.236
  • Finish rate on RacingMania validation tracks
    self-reported
    0.827
  • SWS on RacingMania training sample tracks
    self-reported
    1.250
  • Finish rate on RacingMania training sample tracks
    self-reported
    0.839
  • SWS on RacingMania own training tracks
    self-reported
    1.250
  • Finish rate on RacingMania own training tracks
    self-reported
    0.839