Evaluating these checkpoints in the CS2 environment

Each model is scored by playing a task: the harness drives the live game with the policy and reports score = mean total episode reward (server-authoritative kills/damage minus penalties), plus success rate. This doc is self-contained — pick a checkpoint, arrange it as a run dir, start the game, run one command.

What you need

The gamevla codebase (contains gamevla.envs.cs2 + gamevla.envs.eval + examples/CS2/...) and its conda env (PyTorch, gymnasium, websockets, the CS2 frameworks).
A CS2 dedicated server for the scenario + a rendered game client on the eval machine (Linux with an NVIDIA GPU-backed X display, or Windows). The env owns screen capture + input.
For aimflow*: the Qwen3-VL base model available to the model server.

1. Arrange a checkpoint as a run dir

Un-normalization reads config.yaml + dataset_statistics.json from two dirs above the weight (run_dir = ckpt.parents[1]). Each folder here ships all three, so lay them out as:

run/<exp>/
  config.yaml
  dataset_statistics.json
  checkpoints/<weight>.pt        # e.g. steps_55000.pt  (or final_model.pt)

e.g.

mkdir -p run/aimflow_v3/checkpoints
cp aimflow_v3/config.yaml aimflow_v3/dataset_statistics.json run/aimflow_v3/
cp aimflow_v3/steps_55000.pt run/aimflow_v3/checkpoints/

2. Bring up the game (once)

export PYTHONPATH=<gamevla_repo>:<gamevla_repo>/gamevla/train/starVLA
python -m gamevla.envs.cs2.server.lifecycle up 5e_mirage_prefire --timeout 150
python -m gamevla.envs.cs2.client.connect <host:port> --timeout 600

3. Smoke-test the loop (no model)

Confirms server/client/capture/input/reward all work before wiring a model:

python -m gamevla.envs.eval.run_eval \
  --env cs2/5e_mirage_prefire/connector_to_a_site --endpoint <host:port> \
  --policy idle --episodes 1

4. Run a model

The eval process auto-launches a local model server for the checkpoint, then plays:

python -m gamevla.envs.eval.run_eval \
  --env cs2/5e_mirage_prefire/connector_to_a_site --endpoint <host:port> \
  --policy examples.CS2.aimflow_v3.evaluation.policy:build_policy \
  --policy-kwargs '{"ckpt":"run/aimflow_v3/checkpoints/steps_55000.pt","port":10093,"sensitivity":1.0}' \
  --episodes 3 --bot-count 5 --output result.json
# -> RESULT {... "score": <mean total reward>, "success_rate": ...}

Per-method factories + defaults: examples.CS2.<exp>.evaluation.policy:build_policy for <exp> in csbc, nitrogen_starvla, aimflow, aimflow_v2, aimflow_v3. See each folder's EVAL.md.

Mouse conversion (auto-derived, DPI-free)

Model mouse output is a screen fraction where 1.0 = 45° of view rotation. In CS2 with raw input, Δdeg = counts · sensitivity · m_yaw, so the harness derives mouse_gain = 45 / (sensitivity · m_yaw) (m_yaw 0.022) and pins the matching client cvars (m_rawinput 1; m_customaccel 0; m_yaw 0.022; m_pitch 0.022; sensitivity <S>) at each reset — no DPI, no hand-tuning. Just pass your in-game sensitivity (default 1.0). Sanity-check one flick on the box; if mirrored, add "invert_dx":true / "invert_dy":true.

Agent interface (to add your own policy)

Anything implementing this plugs into evaluate(env, policy, ...):

class Policy:
    def reset(self): ...                       # clear history at episode start
    def act(self, obs, info) -> action_dict    # RGB frame -> env action Dict

Env action space: {keys: MultiBinary(87), buttons: MultiBinary(5), mouse: int32(2), wheel: int32(1)}.

Tasks

Any cs2/<scenario>/<task> id works with --env: cs2/5e_dust2_prefire/{a_long_to_a_site, a_short_to_a_site, a_short_to_top_mid, ct_spawn_to_a_site, ct_spawn_to_top_mid, lower_to_upper_tunnels, lower_tunnels_to_b_site, upper_tunnels_to_b_site} and cs2/5e_mirage_prefire/{a1_to_ct, b_apartments_to_b_site, b_short_to_b_site, connector_to_a_site, ct_to_a_site, market_to_b_site, mid_to_b_short, palace_to_a_site}.

Windows vs Linux

The env auto-selects the transport (Linux mss + /dev/uinput/XTEST; Windows dxcam + Win input), so the same run_eval command works on both. On Windows, start the dedicated server in the interactive desktop user's session and turn off HDR/Advanced Color before capture.