Cortex — compact behavioral-cloning Quake policy

Cortex is a compact specialist game policy: 11M trainable parameters (10.98M exactly) over frozen DINOv3 ViT-S+/16 features. Trained with pure behavior cloning on the ≈475-hour Quake subset of the public Pixels2Play corpus on a single consumer GPU (RTX 5080, 16 GB), it reaches deeper engine-verified route progress on Quake E1M1 than the released P2P-150M and NitroGen (≈500M) generalist gaming agents evaluated under the same protocol.

The Video Preview on this page shows a rollout of this checkpoint on E1M1 from a fresh spawn; there is another run on YouTube.

In one sentence: 4 frames × 41 frozen DINOv3 tokens → a 6-layer 384-d transformer → 36 held-state logits (sampled at temperature 1) plus continuous mouse dx/dy, at 10 Hz. The full architecture, action-schema contract, and design rationale live in the cortex-actor repository; complete results, evaluation protocol, and negative results are in the paper.

Files

• cortex_quake_bc.pt — release checkpoint (fp32, 43.9 MB): model weights, architecture arguments, and the embedded action schema. The frozen DINOv3 encoder (28.7M parameters, ≈39.7M total executed per decision) is not included; it downloads separately from Meta under its own license.
• replay.mp4 — a recorded episode of this checkpoint playing E1M1 (rendered in the Video Preview panel).
• config.json — machine-readable architecture arguments and action schema (mirrors what the checkpoint stores).

Usage

pip install git+https://github.com/kvark/cortex-actor

from cortex_actor import from_hub

model, ck = from_hub("mad-bot/cortex")

# Per step: cls (B, T, 384) and patches (B, T, 5, 8, 384) from frozen
# DINOv3 ViT-S+/16 at 640x400, T = 4 most recent frames at 10 Hz.
out = model(cls, patches=patches)
out["held_logits"]          # (B, 36) — sample at temperature 1
out["mouse_dx"], out["mouse_dy"]  # tanh outputs; scale by 500 / 250 mickeys

The action-channel contract (key roster order, held-state layout, mouse scales) is defined in cortex_actor/schema.py and is part of the architecture: the index of a key in KEYS is its channel in the logits tensor.

Results

On Quake E1M1 under an engine-verified evaluation protocol (N=20 stochastic episodes across 5 seeds, independently replicated), Cortex passes the opening door–button–gate sequence 20/20 and reaches route waypoint median 5–6 (max 9) with 28–32 kills per batch; the released P2P-150M and NitroGen baselines, run in the same environment with their official inference code at matched duration, stall at route waypoint median 1. No system, including ours, completes the level. Full numbers, controls, robustness batteries, and what did not work: the paper.

Authors

Dzmitry Malyshau. Development was heavily AI-assisted by Anthropic's Claude and OpenAI's Codex — see the paper's AI assistance disclosure.

License

MIT for the model weights and code. The DINOv3 encoder is distributed by Meta under its own license; the Pixels2Play corpus belongs to Elefant AI.