README.md

---
license: mit
library_name: pytorch
pipeline_tag: reinforcement-learning
tags:
- behavior-cloning
- imitation-learning
- game-playing
- quake
- dinov3
datasets:
- elefantai/p2p-full-data
base_model: facebook/dinov3-vits16plus-pretrain-lvd1689m
base_model_relation: adapter
---

# Cortex — compact behavioral-cloning Quake policy

Cortex is a compact specialist game policy: **11M trainable parameters** (10.98M exactly) over frozen [DINOv3 ViT-S+/16](https://huggingface.co/facebook/dinov3-vits16plus-pretrain-lvd1689m) features. Trained with pure behavior cloning on the ≈475-hour Quake subset of the public [Pixels2Play corpus](https://huggingface.co/datasets/elefantai/p2p-full-data) on a single consumer GPU (RTX 5080, 16 GB), it reaches deeper engine-verified route progress on Quake E1M1 than the released [P2P-150M](https://huggingface.co/elefantai/open-p2p) and [NitroGen](https://huggingface.co/nvidia/NitroGen) (≈500M) generalist gaming agents evaluated under the same protocol.

The Video Preview on this page shows a rollout of this checkpoint on E1M1 from a fresh spawn; there is [another run on YouTube](https://youtu.be/Ou9NAmFoCOM).

In one sentence: 4 frames × 41 frozen DINOv3 tokens → a 6-layer 384-d transformer → 36 held-state logits (sampled at temperature 1) plus continuous mouse dx/dy, at 10 Hz. The full architecture, action-schema contract, and design rationale live in the [cortex-actor repository](https://github.com/kvark/cortex-actor); complete results, evaluation protocol, and negative results are in the [paper](https://github.com/kvark/cortex-actor/blob/main/paper/cortex_quake_bc.md).

## Files

- `cortex_quake_bc.pt` — release checkpoint (fp32, 43.9 MB): model weights, architecture arguments, and the embedded action schema. The frozen DINOv3 encoder (28.7M parameters, ≈39.7M total executed per decision) is **not** included; it downloads separately from Meta under its own license.
- `replay.mp4` — a recorded episode of this checkpoint playing E1M1 (rendered in the Video Preview panel).
- `config.json` — machine-readable architecture arguments and action schema (mirrors what the checkpoint stores).

## Usage

```bash
pip install git+https://github.com/kvark/cortex-actor
```

```python
from cortex_actor import from_hub

model, ck = from_hub("mad-bot/cortex")

# Per step: cls (B, T, 384) and patches (B, T, 5, 8, 384) from frozen
# DINOv3 ViT-S+/16 at 640x400, T = 4 most recent frames at 10 Hz.
out = model(cls, patches=patches)
out["held_logits"]          # (B, 36) — sample at temperature 1
out["mouse_dx"], out["mouse_dy"]  # tanh outputs; scale by 500 / 250 mickeys
```

The action-channel contract (key roster order, held-state layout, mouse scales) is defined in [`cortex_actor/schema.py`](https://github.com/kvark/cortex-actor/blob/main/cortex_actor/schema.py) and is part of the architecture: the index of a key in `KEYS` is its channel in the logits tensor.

## Results

On Quake E1M1 under an engine-verified evaluation protocol (N=20 stochastic episodes across 5 seeds, independently replicated), Cortex passes the opening door–button–gate sequence 20/20 and reaches route waypoint median 5–6 (max 9) with 28–32 kills per batch; the released [P2P-150M](https://huggingface.co/elefantai/open-p2p) and [NitroGen](https://huggingface.co/nvidia/NitroGen) baselines, run in the same environment with their official inference code at matched duration, stall at route waypoint median 1. No system, including ours, completes the level. Full numbers, controls, robustness batteries, and what did *not* work: [the paper](https://github.com/kvark/cortex-actor/blob/main/paper/cortex_quake_bc.md).

## Authors

Dzmitry Malyshau. Development was heavily AI-assisted by Anthropic's Claude and OpenAI's Codex — see the paper's AI assistance disclosure.

## License

MIT for the model weights and code. The DINOv3 encoder is distributed by Meta under its own license; the Pixels2Play corpus belongs to Elefant AI.