| --- |
| license: apache-2.0 |
| tags: [chess, reinforcement-learning, grpo] |
| --- |
| |
| # Chess-RL-Models |
|
|
| RL (GRPO) post-training checkpoint trajectories for the chess pre-to-post |
| compute-allocation study. Companion to |
| [Chess-Pretrain-Models](https://huggingface.co/pavelslab-nyu/Chess-Pretrain-Models) |
| (pretraining bases) and |
| [Chess-SFT-Models](https://huggingface.co/pavelslab-nyu/Chess-SFT-Models) |
| (SFT initialisations). Model names match across all three repos, so |
| `model_50m_17B` here is the RL run started from `model_50m_17B` there. |
|
|
| **28 models, 680 checkpoints.** Steps kept: every step ≡ 0 mod 100, |
| plus each run's final step. |
|
|
| ## Layout |
|
|
| ``` |
| model_{size}_{pretraining_tokens}/ |
| └── global_step_{N}/ |
| ├── config.json |
| ├── model.safetensors |
| ├── tokenizer.py # custom tokenizer -> trust_remote_code=True |
| ├── vocab.json |
| └── ... |
| ``` |
|
|
| ## Loading |
|
|
| The remote-code resolver ignores `subfolder=`, so download the step folder and |
| load the **local path** (each step folder is self-contained): |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| name, step = "model_50m_17B", "global_step_2000" |
| p = snapshot_download("Pre2Post-Chess-RL/Chess-RL-Models", allow_patterns=f"{name}/{step}/*") + f"/{name}/{step}" |
| model = AutoModelForCausalLM.from_pretrained(p, trust_remote_code=True) |
| tok = AutoTokenizer.from_pretrained(p, trust_remote_code=True) |
| ``` |
|
|
| ## Models |
|
|
| | model | size | pretrain tokens | ckpts | steps | |
| |---|---|---|---|---| |
| | `model_200m_0.27B` | 200m | 0.267B | 10 | 100–1000 | |
| | `model_200m_1.1B` | 200m | 1.07B | 20 | 100–2000 | |
| | `model_200m_2.1B` | 200m | 2.13B | 20 | 100–2000 | |
| | `model_200m_4.0B` | 200m | 4B | 10 | 100–1000 | |
| | `model_200m_5.3B` | 200m | 5.33B | 20 | 100–2000 | |
| | `model_20m_0.53B` | 20m | 0.527B | 20 | 100–2000 | |
| | `model_20m_1.6B` | 20m | 1.58B | 50 | 100–5000 | |
| | `model_20m_2.6B` | 20m | 2.64B | 49 | 100–4900 | |
| | `model_20m_5.3B` | 20m | 5.27B | 50 | 100–5000 | |
| | `model_20m_11B` | 20m | 10.5B | 50 | 100–5000 | |
| | `model_20m_16B` | 20m | 15.8B | 30 | 100–3000 | |
| | `model_20m_21B` | 20m | 21.1B | 50 | 100–5000 | |
| | `model_20m_32B` | 20m | 31.6B | 50 | 100–5000 | |
| | `model_20m_40B` | 20m | 39.6B | 49 | 100–4900 | |
| | `model_20m_53B` | 20m | 52.7B | 32 | 100–5000 | |
| | `model_50m_0.23B` | 50m | 0.229B | 20 | 100–2000 | |
| | `model_50m_0.69B` | 50m | 0.687B | 12 | 100–1200 | |
| | `model_50m_1.1B` | 50m | 1.15B | 10 | 100–1000 | |
| | `model_50m_2.3B` | 50m | 2.29B | 20 | 100–2000 | |
| | `model_50m_4.6B` | 50m | 4.58B | 20 | 100–2000 | |
| | `model_50m_9.2B` | 50m | 9.16B | 20 | 100–2000 | |
| | `model_50m_17B` | 50m | 17.2B | 20 | 100–2000 | |
| | `model_50m_23B` | 50m | 22.9B | 10 | 100–1000 | |
| | `model_50m_41B` | 50m | 41.2B | 18 | 100–2000 | |
| | `model_680m_0.32B` | 680m | 0.319B | 5 | 100–500 | |
| | `model_680m_1.6B` | 680m | 1.6B | 1 | 2000 | |
| | `model_680m_6.4B` | 680m | 6.4B | 10 | 100–1000 | |
| | `model_680m_16B` | 680m | 16B | 4 | 1200–1900 | |
|
|
| ## Not included |
|
|
| These models exist in the study but had no usable weights available at upload |
| time: |
|
|
| - `model_200m_0.53B` |
| - `model_200m_11B` |
| - `model_200m_21B` |
| - `model_200m_40B` |
| - `model_200m_53B` |
| - `model_680m_0.64B` |
| - `model_680m_1.2B` |
| - `model_680m_12B` |
| - `model_680m_3.2B` |
| - `model_680m_32B` |
| - 2 32m models absent from the pretraining-token tables |
|
|
