davidafrica/chess-e2h-rl-g16

Chess easy-to-hard RL checkpoints (group size 16)

RL checkpoints from the chess easy-to-hard experiment (code at commit 1e78e48). Each run starts from a chess transformer (20M, 100M or 200M parameters) that was pre-trained on human games and then SFT'd on chain-of-thought solutions to 800–900 Elo puzzles (SFT checkpoints for 20M, 100M and 200M, one per pre-training budget). It is then trained with GRPO in Miles on these puzzles (900_easy17k_1200_hard): 64 puzzles × 16 samples per step, 2,000 steps.

Folder	Trains on	Reward
all_rb_g16	easy + hard	ground truth on both
easy_rb_g16	easy only	ground truth
mixed_rb_mv_g16	easy + hard	ground truth on easy, majority vote on hard
hard_mv_g16	hard only	majority vote

Easy puzzles are rated 800–900 Elo and hard ones 900–1200. Majority vote rewards a sample for matching the most common answer among the puzzle's 16 samples, so the ground truth is never seen.

Layout

<size>_<budget>/<arm>_g16/global_step_<N>/, where <size> is 20m, 100m or 200m, <budget> is the pre-training budget in tokens and N runs from 100 to 2000 in steps of 100. Each folder is a Hugging Face model (Qwen3ForCausalLM, fp32) with the chess tokenizer (tokenizer.py, vocab.json). COMMITTED.json lists each file's SHA-256.

Budgets:

• 100M: 40B, 20B, 11B, 2.1B, 1.1B and 0.53B, all complete.
• 20M: 53B, 32B, 11B, 2.6B, 1.6B and 0.53B, all complete.
• 200M: 40B, 21B, 11B, 5.3B, 1.1B and 0.53B, all complete.

Resuming past step 2000

Each arm also keeps its final training state, so a run can continue past step 2000. <size>_<budget>/<arm>_g16/trajectory_sep_no_labels/multi_turn_lr1e-5_bs1024_kl0.001_res2560_minimax_grpo_miles/<size>_<budget>/checkpoints/ holds latest_checkpointed_iteration.txt and iter_0002000/ (model, optimizer, LR scheduler, RNG and rollout state, as torch distributed checkpoints saved by 2 GPUs). The path matches the run's layout under OUTPUT_DIR, so downloading it there puts it where run_rl.sh looks with RESUME=1:

hf download davidafrica/chess-e2h-rl-g16 --local-dir "$OUTPUT_DIR" \
  --include '100m_40B/all_rb_g16/trajectory_sep_no_labels/*'