mad-bot commited on
Commit
42dc2e5
·
verified ·
1 Parent(s): 9c8cc75

Slim the card, fix strikethrough, link baselines, embed rollout video

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +8 -6
  3. rollout.mp4 +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ rollout.mp4 filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -11,21 +11,23 @@ tags:
11
  datasets:
12
  - elefantai/p2p-full-data
13
  base_model: facebook/dinov3-vits16plus-pretrain-lvd1689m
 
14
  ---
15
 
16
  # Cortex — compact behavioral-cloning Quake policy
17
 
18
- Cortex is a compact behavioral-cloning game policy: **10.98M trainable parameters** over frozen [DINOv3 ViT-S+/16](https://huggingface.co/facebook/dinov3-vits16plus-pretrain-lvd1689m) features (39.7M parameters executed per decision including the frozen encoder). Trained with pure behavior cloning on the ~475-hour Quake subset of the public [Pixels2Play corpus](https://huggingface.co/datasets/elefantai/p2p-full-data) on a single consumer GPU (RTX 5080, 16 GB), it reaches deeper engine-verified route progress on Quake E1M1 than the released P2P-150M and NitroGen (~500M) generalist gaming agents evaluated under the same protocol.
19
 
20
- Model code and action schema: **[github.com/kvark/cortex-actor](https://github.com/kvark/cortex-actor)**. Watch this checkpoint play E1M1: **[youtu.be/Ou9NAmFoCOM](https://youtu.be/Ou9NAmFoCOM)**.
21
 
22
- ## Architecture
23
 
24
- Per 100 ms decision: each 640×400 frame is encoded by frozen DINOv3 into a CLS token plus an ordered 5×8 spatial sample of the 25×40 patch grid (41 tokens × 384-d). The last 4 frames (300 ms) pass through a 6-layer, 384-d, 6-head bidirectional transformer encoder; the last frame's CLS position feeds two heads: 36 independent held-state logits (33 keys + 3 mouse buttons, absolute state, temperature-1 sampled at deploy) and tanh-squashed continuous mouse dx/dy. There is no previous-action input, pose, map, text, auxiliary loss, or game-specific rule.
25
 
26
  ## Files
27
 
28
- - `cortex_quake_bc.pt` — release checkpoint (fp32, 43.9 MB): model weights, architecture arguments, and the embedded action schema (active keys, held-state channel layout, training-alignment constants). The frozen DINOv3 encoder is **not** included; download it separately from Meta under its own license.
 
29
 
30
  ## Usage
31
 
@@ -55,7 +57,7 @@ The action-channel contract (key roster order, held-state layout, mouse scales)
55
 
56
  ## Results
57
 
58
- On Quake E1M1 under an engine-verified evaluation protocol (completion counted only on the engine's level-transition event; N=20 stochastic episodes across 5 seeds, independently replicated), Cortex passes the opening door–button–gate sequence 20/20 and reaches route waypoint median 5–6 (max 9) with 28–32 kills per batch. The released P2P-150M and NitroGen baselines, run in the same environment with their official inference code and published settings at matched duration, stall at route waypoint median 1. Results generalize across additional maps and shared mid-map start states. No system, including ours, completes the level. Full numbers, controls, and negative results are in the paper: [Cortex: An 11M-Parameter Specialist Policy Outperforms Foundation-Scale Gaming Agents on Quake](https://github.com/kvark/cortex-actor/blob/main/paper/cortex_quake_bc.md).
59
 
60
  ## Authors
61
 
 
11
  datasets:
12
  - elefantai/p2p-full-data
13
  base_model: facebook/dinov3-vits16plus-pretrain-lvd1689m
14
+ base_model_relation: adapter
15
  ---
16
 
17
  # Cortex — compact behavioral-cloning Quake policy
18
 
19
+ Cortex is a compact specialist game policy: **11M trainable parameters** (10.98M exactly) over frozen [DINOv3 ViT-S+/16](https://huggingface.co/facebook/dinov3-vits16plus-pretrain-lvd1689m) features. Trained with pure behavior cloning on the 475-hour Quake subset of the public [Pixels2Play corpus](https://huggingface.co/datasets/elefantai/p2p-full-data) on a single consumer GPU (RTX 5080, 16 GB), it reaches deeper engine-verified route progress on Quake E1M1 than the released [P2P-150M](https://huggingface.co/elefantai/open-p2p) and [NitroGen](https://huggingface.co/nvidia/NitroGen) (500M) generalist gaming agents evaluated under the same protocol.
20
 
21
+ <video controls src="https://huggingface.co/mad-bot/cortex/resolve/main/rollout.mp4"></video>
22
 
23
+ *This checkpoint playing E1M1 from a fresh spawn ([full run on YouTube](https://youtu.be/Ou9NAmFoCOM)).*
24
 
25
+ In one sentence: 4 frames × 41 frozen DINOv3 tokens a 6-layer 384-d transformer 36 held-state logits (sampled at temperature 1) plus continuous mouse dx/dy, at 10 Hz. The full architecture, action-schema contract, and design rationale live in the [cortex-actor repository](https://github.com/kvark/cortex-actor); complete results, evaluation protocol, and negative results are in the [paper](https://github.com/kvark/cortex-actor/blob/main/paper/cortex_quake_bc.md).
26
 
27
  ## Files
28
 
29
+ - `cortex_quake_bc.pt` — release checkpoint (fp32, 43.9 MB): model weights, architecture arguments, and the embedded action schema. The frozen DINOv3 encoder (28.7M parameters, ≈39.7M total executed per decision) is **not** included; it downloads separately from Meta under its own license.
30
+ - `rollout.mp4` — a recorded episode of this checkpoint playing E1M1.
31
 
32
  ## Usage
33
 
 
57
 
58
  ## Results
59
 
60
+ On Quake E1M1 under an engine-verified evaluation protocol (N=20 stochastic episodes across 5 seeds, independently replicated), Cortex passes the opening door–button–gate sequence 20/20 and reaches route waypoint median 5–6 (max 9) with 28–32 kills per batch; the released [P2P-150M](https://huggingface.co/elefantai/open-p2p) and [NitroGen](https://huggingface.co/nvidia/NitroGen) baselines, run in the same environment with their official inference code at matched duration, stall at route waypoint median 1. No system, including ours, completes the level. Full numbers, controls, robustness batteries, and what did *not* work: [the paper](https://github.com/kvark/cortex-actor/blob/main/paper/cortex_quake_bc.md).
61
 
62
  ## Authors
63
 
rollout.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae68d08d6bb8facac3d570b409ad5ebcd7984dafc566ee2efc655d40a9060463
3
+ size 21394993