racingmania-pqn-seed1

PQN, seed 1: PQN baseline

Status: evaluated (evaluated; scores available); physics version v1; card generated 2026-11-15 from the result files.

RacingMania is an original 3D time-attack racing game for the browser (three.js + cannon-es RaycastVehicle) with reinforcement-learning drivers trained on a batched PyTorch port of the same car physics. Code, paper and full results: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Videos

Official evaluation runs of this model replayed in the browser game ("Watch replay"): the car states the evaluator recorded while scoring the run, so each clip is exactly the run its caption describes. Clip numbers are those of the README, which has more clips, including the controls: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game#videos.

Best lap of the PQN baseline

  • What you see: The baseline algorithm, PQN (parallelised Q-learning), trained with the same inputs, actions, reward and step budget as the PPO generalist on generated tracks. The track was never part of its training.
  • Model: pqn/seed1: PQN, trained by reinforcement learning for 1 billion agent steps on 48,949 generated training tracks (seed 1, 1.1 h on one GPU).
  • Map: Technical (campaign-04), a hand-made campaign track, 666 m long with no jumps and up to 5° of banking; this model never saw it during training.
  • Evaluation: official evaluation in results/pqn/seed1/eval_campaign.csv (standing start, most probable action, final checkpoint): finished in 26.04 s; the scripted driver needs 42.52 s, so this is 1.63× its speed. The clip replays this run.
  • Why this clip: chosen by a fixed rule as the highest-scoring finished track of the 5 campaign tracks it finished in its official evaluation.

Model

Item Value
Algorithm PQN
Architecture Q-network 2x256, LayerNorm, one 9-way head
Objective centreline progress
Observation car state + 20 look-ahead points
Training data 48,949 generated tracks
Start states standing + random centreline (p=0.5)
Agent steps trained 1,000,341,504
Wall-clock hours 1.10
Git commit 70a35809b494c89e7fa074064a9c8dc961ef0360

Evaluation

Success-weighted speed (SWS): mean over tracks of finished x (scripted driver's time / agent's time); 1.0 means finishing every track as fast as the scripted reference driver. One deterministic run per track from a standing start with the most probable action; intervals are 95 % percentile bootstrap intervals over tracks (one seed).

Track set Tracks SWS [95 % CI] Finished
Campaign (hand-made) 5 1.391 [1.270, 1.517] 100.0 %
Out-of-range test 187 0.725 [0.631, 0.820] 55.1 %
Test (unseen generated) 971 1.054 [1.020, 1.086] 79.8 %
Unfiltered test 200 1.105 [1.035, 1.173] 83.0 %
Own training tracks 1000 1.052 [1.019, 1.085] 79.7 %
Training sample 1000 1.052 [1.019, 1.085] 79.7 %
Validation 197 1.034 [0.956, 1.109] 78.2 %

How to use

Files: final.pt (PyTorch checkpoint: agent and norm state dicts, ppo and env_cfg settings, obs_dim), policy.onnx (observation function + frozen normaliser + actor in one graph, as run by the browser game), run.json, training and validation curves, and per-track evaluation CSVs under eval/.

PyTorch, inside a clone of the repository (uv sync):

PQN checkpoints hold a Q-network; load them with racingmania.rl.pqn (see the repository).

ONNX (inputs and outputs are specified in docs/formats.md, section "Policy model"):

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("policy.onnx")
inputs = {i.name: np.zeros([d if isinstance(d, int) else 1 for d in i.shape], np.float32) for i in sess.get_inputs()}
steer_logits, pedal_logits = sess.run(None, inputs)
steer, pedal = int(steer_logits.argmax()), int(pedal_logits.argmax())  # 0/1/2 = left/none/right, gas/coast/brake

To watch it drive, open the game (web/index.html, or the GitHub Pages site), choose it in a track's AI drivers table and press "Watch AI" (live in the browser) or "Watch replay" (its official evaluation run).

Training configuration (from run.json)

Setting Value
total_steps 1000000000
num_envs 16384
num_steps 32
lr 0.00025
adaptive_lr False
desired_kl 0.01
lr_min 1e-05
lr_max 0.01
gamma 0.99
gae_lambda 0.95
num_minibatches 4
update_epochs 4
clip_coef 0.2
ent_coef 0.01
vf_coef 1.0
max_grad_norm 10.0
weight_decay 0.0
hidden [256, 256]
action_mode joint
eval_every_minutes 10.0
checkpoint_every_minutes 10.0
rotate_tracks_every 20
loaded_tracks 4096
seed 1
compile True
q_lambda 0.65
start_e 1.0
end_e 0.05
exploration_fraction 0.5
env.random_start_prob 0.5
env.random_start_speed [0.0, 30.0]
env.random_start_lateral 0.5
env.max_steps 1500
env.reward_scale 0.1
env.window_k 20
env.window_stride 3
env.blind False
env.time_cost 0.0
env.fall_penalty 0.0
env.hint_weight 0.0
env.scripted_starts False

Licence and citation

MIT. Author: Libor "Burny" Burian. Repository: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Collection including BurnyCoder/racingmania-pqn-seed1

Evaluation results

  • SWS on RacingMania test (unseen generated) tracks
    self-reported
    1.054
  • Finish rate on RacingMania test (unseen generated) tracks
    self-reported
    0.798
  • SWS on RacingMania validation tracks
    self-reported
    1.034
  • Finish rate on RacingMania validation tracks
    self-reported
    0.782
  • SWS on RacingMania training sample tracks
    self-reported
    1.052
  • Finish rate on RacingMania training sample tracks
    self-reported
    0.797
  • SWS on RacingMania own training tracks
    self-reported
    1.052
  • Finish rate on RacingMania own training tracks
    self-reported
    0.797