README.md

---
title: Agent Parkour
emoji: 🏃
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.18.0
python_version: 3.12
app_file: app.py
fullWidth: true
pinned: true
license: other
license_name: "MIT code + third-party asset licenses"
short_description: 3D parkour maps and races with a tiny RL agent.
tags:
  - track:wood
  - achievement:offbrand
  - achievement:fieldnotes
---

# Agent Parkour

Agent Parkour is a browser-playable 3D parkour game where generated
floating-platform courses become tests for a trained reinforcement learning
agent. A player can build a course with procedural controls or with the built-in
Gemma 4 31B map mode. The trained agent is then dispatched in simulation, its
best clear becomes a replay, and the player races that run on the same map.

## TL;DR For Judges

| Field | Details |
| --- | --- |
| Track | Thousand Token Wood |
| What it does | Generate a 3D parkour map either with an LLM or procedurally, dispatch a trained reinforcement learning agent to run it, then race against the agent on that same course. |
| Map creation | Procedural generation by default, or optional LLM course drafting with the built-in Gemma 4 31B map mode. |
| Small model | The game agent has 866,057 learned parameters. It is an end-to-end neural controller that reads egocentric movement, goal, and platform observations, then outputs steering, turning, jump, and sprint actions. |
| Demo video | https://www.youtube.com/watch?v=cAzgDIMmTf8 |
| Social post | https://x.com/marmcisgreat/status/2066669787087036785 |
| Space | https://huggingface.co/spaces/build-small-hackathon/agent-parkour |

## Directory Structure

| Path | Purpose |
| --- | --- |
| `app.py` | Root Space entrypoint. Loads and launches the Gradio server. |
| `app/app.py` | Public app server, map-generation controls, LLM map path, agent rollout loop, saved maps, and frontend serving. |
| `app/frontend/` | React/Three.js game frontend: map builder, replay viewer, and playable race mode. |
| `env.py` | Pure Torch parkour simulation: movement physics, observations, rewards, landings, falling, and completion. |
| `runner.py` | Reinforcement learning agent architectures. The active model is the token-attention controller. |
| `mapgen.py` | Procedural route motifs, distractor placement, trap placement, reachability checks, and geometry repair. |
| `train.py`, `infer.py`, research scripts | Training and evaluation code used to build and inspect the agent. This code is included for transparency; the public app uses the trained model. |
| `settings.py`, `config.py`, `runtime.py` | Runtime defaults, model path, physics constants, and device selection. |
| `replay.py` | Replay serialization utilities. |

## Reinforcement Learning Agent

The agent is the core technical experiment: can a neural controller learn to
solve 3D parkour puzzles through long sequences of movement decisions? It is
trained end-to-end inside a custom 3D parkour simulation using reinforcement
learning. The submitted behavior uses a controller that reads local state and
directly outputs movement actions.

A good run requires route solving as well as movement control. Some platforms
act as bait, some routes bend away from the obvious goal direction, some jumps
require committing before the goal is visually convenient, and some attractive
branches lead to failure. The agent has to keep making useful decisions across
many timesteps. Executing a single isolated jump well is only a small part of
the problem.

The observation design gives the agent an egocentric view of the course. The
model receives local velocity, grounded state, the final goal as a local beacon,
yaw, time remaining, and nearby platform tokens. Each platform token describes a
platform relative to the agent's current position, so the route has to be read
during the run from the agent's point of view.

The current agent uses a 188-number observation: 12 base values plus 16
nearest-platform tokens with 11 values each, so `12 + (16 * 11) = 188`. The
token tensor reserves a few internal platform-state bits, while the public map
modes in this submission are driven by geometry, goal visibility, and the
agent's movement state.

```text
base =
  local forward / strafe / vertical velocity, divided by [6, 6, 8]
  grounded flag
  final-goal local x/y divided by 24
  final-goal local z divided by 4
  final-goal xy distance divided by 24
  sin(yaw)
  cos(yaw)
  time remaining
  progress slot, disabled for the current agent and therefore 0

platform token summary, repeated for the 16 nearest visible platforms =
  hit flag
  local platform x divided by 9.5 sensor range
  local platform y divided by 9.5 sensor range
  local platform z divided by 3
  platform width divided by 1.7
  platform depth divided by 1.5
  platform distance divided by 9.5 sensor range
  is final goal
  reserved internal platform-state bits
```

The active model uses 16 nearest-platform tokens, distance sorting, a 120 degree
field of view, and disabled progress observation. It outputs five actions:
forward/back, strafe, turn, jump, and sprint. Jump is applied through the same
grounding rules used by the game physics.

The reward design makes the training objective explicit. Under the current
public map settings, the active terms reward distance-to-goal progress, first
visits to new platforms, reaching the goal, and finishing quickly. Falling,
timeouts, stagnation, and extra steps are penalized. In code, the environment
reward is:

```text
reward =
  progress_delta * goal_progress_reward_scale
  + first_visit * first_visit_reward
  + unlocked_now * unlock_reward
  + on_goal * 20.0
  + on_goal * success_time_bonus * clamp(1 - steps / max_steps, 0, 1)
  - fallen * fall_penalty
  - timeout * 2.0
  - stagnation_timeout * stagnation_penalty
  - active * step_cost
```

This is the shared environment reward formula. In the submitted maps,
`unlock_platforms` is off, `stagnation_patience_steps` is 0, and
`stagnation_penalty` is 0, so the unlock and stagnation-penalty terms contribute
zero. The active terms are goal progress scaled by `1.35`, first visits to new
platforms worth `0.12`, a goal reward of `20.0`, a small finish-time bonus of
`0.06` times the remaining-time fraction, a fall penalty of `4.0`, a timeout
penalty of `2.0`, and a per-step cost of `0.0005`. This gives the agent steady
feedback for moving toward the goal, separate credit for discovering new
platforms, and a much larger terminal reward for actually clearing the course.

Training used large batches of parallel Torch environments. Each curriculum
stage increases route length and distractor count, collects rollouts from the
current policy, computes discounted advantages, then updates the policy with
clipped policy loss, value loss, entropy regularization, gradient clipping, and
self-imitation on successful trajectories. The active training run used top-k
platform observations, distance sorting, hidden size 256, and curriculum stages
that reached long maps with many distractors. The training and research scripts
remain in the repo as auditable research artifacts; the public product path uses
the trained model file.

## Procedural Map Generation

The procedural generator is designed to create maps that are solvable,
deceptive, and physically coherent under the same movement model used by the
agent. It starts by building a main route, then places surrounding platforms
that create alternate paths, traps, and misleading visual structure.

The main route is assembled from rhythm motifs such as crescents, doglegs,
vertical terraces, loopback shelves, S-curves, ladders, and fork/merge bends.
Difficulty controls change how those motifs are selected and scaled. Length,
verticality, jump gap, turniness, precision, platform size, decoys, traps, tiny
platforms, and unintuitive routing all feed into the final course shape.

After the route exists, extra platforms add the puzzle pressure. The generator
can place direct decoys, clusters, false branches, false finishes, false
summits, braid bridges, trap jumps, backtracks, valleys, hairpins, and
greedy-looking branches that can waste a run. Some extras are legal wrong
turns, some are difficult to return from, and some mimic an obvious path toward
the goal while the real route bends elsewhere.

A final repair pass keeps the geometry playable. The generator enforces route
reachability, route edge gaps, overlap separation, route-extra collision
cleanup, extra-platform reachability, and clamping of extras into the route
span. This is what lets the generator produce hard maps while keeping them tied
to the same physics that the trained agent uses.

## LLM Map Generation

The LLM path provides a second source of generated maps while keeping the same
runtime contract as procedural generation. The built-in Gemma 4 31B map mode
writes the course layout directly as compact JSON:

```json
{"platforms":[{"x":0,"y":0,"z":0,"w":1.5,"d":1.5}]}
```

The LLM output is treated as draft geometry. The backend repairs JSON if needed,
clamps values into legal bounds, applies the same reachability and overlap
checks used by procedural maps, and sends the result to the reinforcement
learning agent.

For deployment, we configure the LLM map mode with Space secrets. During local
development, the same settings can come from environment variables:

```bash
MAPGEN_LLM_API_KEY=...
MAPGEN_LLM_MODEL=google/gemma-4-31b-it
MAPGEN_LLM_TIMEOUT=30
```

Procedural generation is the default path. The LLM path is an optional way to
test whether the trained agent can generalize to course layouts outside the
hand-built procedural distribution.

## Game Loop

The game loop is generate, run, replay, and race. The user starts by generating
a map either procedurally or with the LLM. Once the map is built, the trained
reinforcement learning agent is dispatched in the canonical Python simulation.
It runs many attempts in parallel, using the same movement physics and
observation code used during training.

Successful attempts are ranked by completion speed, and the fastest clear is
selected. When every attempt misses the goal, the best-progress attempt is kept
and the result is marked uncertified. The selected rollout is materialized into
replay frames for the frontend, then the player can inspect that run and race
against it in the browser. Collisions are evaluated against the top surfaces of
the parkour platforms; trees, props, and other scenery are visual only.

## Local Development

Install dependencies:

```bash
uv sync --extra app
npm --prefix app/frontend install
```

Build the frontend bundle used by the Space:

```bash
npm --prefix app/frontend run build
```

Run locally:

```bash
PORT=7860 uv run python app.py
```

Open the port exposed by the frontend.

## License And Asset Credits

The project code is intended to be released under the MIT License. Some visual
and game assets are third-party paid assets, so they are covered by their
original marketplace or vendor licenses instead of the project code license.
Those assets are used in this demo under our purchased rights and should be
reused only under the terms from their original sources.

| Asset | Source / License Link | Usage |
| --- | --- | --- |
| Platform/environment models | [KayKit Forest](https://kaylousberg.itch.io/kaykit-forest) | Floating platforms and forest-style game environment pieces |
| Character animations | [KayKit Character Animations](https://kaylousberg.itch.io/kaykit-character-animations) | Player and agent animation clips |
| Character models | [KayKit Adventurers](https://kaylousberg.itch.io/kaykit-adventurers) | Player and agent character visuals |
| Skybox | [Free Stylized Skybox 65](https://freestylized.com/skybox/sky_65/) | Sky/background environment |
| Water texture | [Vecteezy water texture](https://www.vecteezy.com/vector-art/1844212-water-texture-top-view-background-vector-design-illustration) | Water surface visual |