README.md

metadata

title: Agent Parkour
emoji: 🏃
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.18.0
python_version: 3.12
app_file: app.py
fullWidth: true
pinned: true
license: other
license_name: MIT code + third-party asset licenses
short_description: 3D parkour maps and races with a tiny RL agent.
tags:
  - track:wood
  - achievement:offbrand
  - achievement:fieldnotes

Agent Parkour

Agent Parkour is a browser-playable 3D parkour game where generated floating-platform courses become tests for a trained reinforcement learning agent. A player can build a course with procedural controls or with the built-in Gemma 4 31B map mode. The trained agent is then dispatched in simulation, its best clear becomes a replay, and the player races that run on the same map.

TL;DR For Judges

Field	Details
Track	Thousand Token Wood
What it does	Generate a 3D parkour map either with an LLM or procedurally, dispatch a trained reinforcement learning agent to run it, then race against the agent on that same course.
Map creation	Procedural generation by default, or optional LLM course drafting with the built-in Gemma 4 31B map mode.
Small model	The game agent has 866,057 learned parameters. It is an end-to-end neural controller that reads egocentric movement, goal, and platform observations, then outputs steering, turning, jump, and sprint actions.
Demo video	https://www.youtube.com/watch?v=cAzgDIMmTf8
Social post	https://x.com/marmcisgreat/status/2066669787087036785
Space	https://huggingface.co/spaces/build-small-hackathon/agent-parkour

Directory Structure

Path	Purpose
app.py	Root Space entrypoint. Loads and launches the Gradio server.
app/app.py	Public app server, map-generation controls, LLM map path, agent rollout loop, saved maps, and frontend serving.
app/frontend/	React/Three.js game frontend: map builder, replay viewer, and playable race mode.
env.py	Pure Torch parkour simulation: movement physics, observations, rewards, landings, falling, and completion.
runner.py	Reinforcement learning agent architectures. The active model is the token-attention controller.
mapgen.py	Procedural route motifs, distractor placement, trap placement, reachability checks, and geometry repair.
train.py, infer.py, research scripts	Training and evaluation code used to build and inspect the agent. This code is included for transparency; the public app uses the trained model.
settings.py, config.py, runtime.py	Runtime defaults, model path, physics constants, and device selection.
replay.py	Replay serialization utilities.

Reinforcement Learning Agent

The agent is the core technical experiment: can a neural controller learn to solve 3D parkour puzzles through long sequences of movement decisions? It is trained end-to-end inside a custom 3D parkour simulation using reinforcement learning. The submitted behavior uses a controller that reads local state and directly outputs movement actions.

A good run requires route solving as well as movement control. Some platforms act as bait, some routes bend away from the obvious goal direction, some jumps require committing before the goal is visually convenient, and some attractive branches lead to failure. The agent has to keep making useful decisions across many timesteps. Executing a single isolated jump well is only a small part of the problem.

The observation design gives the agent an egocentric view of the course. The model receives local velocity, grounded state, the final goal as a local beacon, yaw, time remaining, and nearby platform tokens. Each platform token describes a platform relative to the agent's current position, so the route has to be read during the run from the agent's point of view.

The current agent uses a 188-number observation: 12 base values plus 16 nearest-platform tokens with 11 values each, so 12 + (16 * 11) = 188. The token tensor reserves a few internal platform-state bits, while the public map modes in this submission are driven by geometry, goal visibility, and the agent's movement state.

base =
  local forward / strafe / vertical velocity, divided by [6, 6, 8]
  grounded flag
  final-goal local x/y divided by 24
  final-goal local z divided by 4
  final-goal xy distance divided by 24
  sin(yaw)
  cos(yaw)
  time remaining
  progress slot, disabled for the current agent and therefore 0

platform token summary, repeated for the 16 nearest visible platforms =
  hit flag
  local platform x divided by 9.5 sensor range
  local platform y divided by 9.5 sensor range
  local platform z divided by 3
  platform width divided by 1.7
  platform depth divided by 1.5
  platform distance divided by 9.5 sensor range
  is final goal
  reserved internal platform-state bits

The active model uses 16 nearest-platform tokens, distance sorting, a 120 degree field of view, and disabled progress observation. It outputs five actions: forward/back, strafe, turn, jump, and sprint. Jump is applied through the same grounding rules used by the game physics.

The reward design makes the training objective explicit. Under the current public map settings, the active terms reward distance-to-goal progress, first visits to new platforms, reaching the goal, and finishing quickly. Falling, timeouts, stagnation, and extra steps are penalized. In code, the environment reward is:

reward =
  progress_delta * goal_progress_reward_scale
  + first_visit * first_visit_reward
  + unlocked_now * unlock_reward
  + on_goal * 20.0
  + on_goal * success_time_bonus * clamp(1 - steps / max_steps, 0, 1)
  - fallen * fall_penalty
  - timeout * 2.0
  - stagnation_timeout * stagnation_penalty
  - active * step_cost

This is the shared environment reward formula. In the submitted maps, unlock_platforms is off, stagnation_patience_steps is 0, and stagnation_penalty is 0, so the unlock and stagnation-penalty terms contribute zero. The active terms are goal progress scaled by 1.35, first visits to new platforms worth 0.12, a goal reward of 20.0, a small finish-time bonus of 0.06 times the remaining-time fraction, a fall penalty of 4.0, a timeout penalty of 2.0, and a per-step cost of 0.0005. This gives the agent steady feedback for moving toward the goal, separate credit for discovering new platforms, and a much larger terminal reward for actually clearing the course.

Training used large batches of parallel Torch environments. Each curriculum stage increases route length and distractor count, collects rollouts from the current policy, computes discounted advantages, then updates the policy with clipped policy loss, value loss, entropy regularization, gradient clipping, and self-imitation on successful trajectories. The active training run used top-k platform observations, distance sorting, hidden size 256, and curriculum stages that reached long maps with many distractors. The training and research scripts remain in the repo as auditable research artifacts; the public product path uses the trained model file.

Procedural Map Generation

The procedural generator is designed to create maps that are solvable, deceptive, and physically coherent under the same movement model used by the agent. It starts by building a main route, then places surrounding platforms that create alternate paths, traps, and misleading visual structure.

The main route is assembled from rhythm motifs such as crescents, doglegs, vertical terraces, loopback shelves, S-curves, ladders, and fork/merge bends. Difficulty controls change how those motifs are selected and scaled. Length, verticality, jump gap, turniness, precision, platform size, decoys, traps, tiny platforms, and unintuitive routing all feed into the final course shape.

After the route exists, extra platforms add the puzzle pressure. The generator can place direct decoys, clusters, false branches, false finishes, false summits, braid bridges, trap jumps, backtracks, valleys, hairpins, and greedy-looking branches that can waste a run. Some extras are legal wrong turns, some are difficult to return from, and some mimic an obvious path toward the goal while the real route bends elsewhere.

A final repair pass keeps the geometry playable. The generator enforces route reachability, route edge gaps, overlap separation, route-extra collision cleanup, extra-platform reachability, and clamping of extras into the route span. This is what lets the generator produce hard maps while keeping them tied to the same physics that the trained agent uses.

LLM Map Generation

The LLM path provides a second source of generated maps while keeping the same runtime contract as procedural generation. The built-in Gemma 4 31B map mode writes the course layout directly as compact JSON:

{"platforms":[{"x":0,"y":0,"z":0,"w":1.5,"d":1.5}]}

The LLM output is treated as draft geometry. The backend repairs JSON if needed, clamps values into legal bounds, applies the same reachability and overlap checks used by procedural maps, and sends the result to the reinforcement learning agent.

For deployment, we configure the LLM map mode with Space secrets. During local development, the same settings can come from environment variables:

MAPGEN_LLM_API_KEY=...
MAPGEN_LLM_MODEL=google/gemma-4-31b-it
MAPGEN_LLM_TIMEOUT=30

Procedural generation is the default path. The LLM path is an optional way to test whether the trained agent can generalize to course layouts outside the hand-built procedural distribution.

Game Loop

The game loop is generate, run, replay, and race. The user starts by generating a map either procedurally or with the LLM. Once the map is built, the trained reinforcement learning agent is dispatched in the canonical Python simulation. It runs many attempts in parallel, using the same movement physics and observation code used during training.

Successful attempts are ranked by completion speed, and the fastest clear is selected. When every attempt misses the goal, the best-progress attempt is kept and the result is marked uncertified. The selected rollout is materialized into replay frames for the frontend, then the player can inspect that run and race against it in the browser. Collisions are evaluated against the top surfaces of the parkour platforms; trees, props, and other scenery are visual only.

Local Development

Install dependencies:

uv sync --extra app
npm --prefix app/frontend install

Build the frontend bundle used by the Space:

npm --prefix app/frontend run build

Run locally:

PORT=7860 uv run python app.py

Open the port exposed by the frontend.

License And Asset Credits

The project code is intended to be released under the MIT License. Some visual and game assets are third-party paid assets, so they are covered by their original marketplace or vendor licenses instead of the project code license. Those assets are used in this demo under our purchased rights and should be reused only under the terms from their original sources.

Asset	Source / License Link	Usage
Platform/environment models	KayKit Forest	Floating platforms and forest-style game environment pieces
Character animations	KayKit Character Animations	Player and agent animation clips
Character models	KayKit Adventurers	Player and agent character visuals
Skybox	Free Stylized Skybox 65	Sky/background environment
Water texture	Vecteezy water texture	Water surface visual