File size: 3,386 Bytes
fa9cc05 f75c74f fa9cc05 f75c74f fa9cc05 f75c74f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | ---
license: apache-2.0
tags: [chess, reinforcement-learning, grpo]
---
# Chess-RL-Models
RL (GRPO) post-training checkpoint trajectories for the chess pre-to-post
compute-allocation study. Companion to
[Chess-Pretrain-Models](https://huggingface.co/pavelslab-nyu/Chess-Pretrain-Models)
(pretraining bases) and
[Chess-SFT-Models](https://huggingface.co/pavelslab-nyu/Chess-SFT-Models)
(SFT initialisations). Model names match across all three repos, so
`model_50m_17B` here is the RL run started from `model_50m_17B` there.
**28 models, 680 checkpoints.** Steps kept: every step ≡ 0 mod 100,
plus each run's final step.
## Layout
```
model_{size}_{pretraining_tokens}/
└── global_step_{N}/
├── config.json
├── model.safetensors
├── tokenizer.py # custom tokenizer -> trust_remote_code=True
├── vocab.json
└── ...
```
## Loading
The remote-code resolver ignores `subfolder=`, so download the step folder and
load the **local path** (each step folder is self-contained):
```python
from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
name, step = "model_50m_17B", "global_step_2000"
p = snapshot_download("Pre2Post-Chess-RL/Chess-RL-Models", allow_patterns=f"{name}/{step}/*") + f"/{name}/{step}"
model = AutoModelForCausalLM.from_pretrained(p, trust_remote_code=True)
tok = AutoTokenizer.from_pretrained(p, trust_remote_code=True)
```
## Models
| model | size | pretrain tokens | ckpts | steps |
|---|---|---|---|---|
| `model_200m_0.27B` | 200m | 0.267B | 10 | 100–1000 |
| `model_200m_1.1B` | 200m | 1.07B | 20 | 100–2000 |
| `model_200m_2.1B` | 200m | 2.13B | 20 | 100–2000 |
| `model_200m_4.0B` | 200m | 4B | 10 | 100–1000 |
| `model_200m_5.3B` | 200m | 5.33B | 20 | 100–2000 |
| `model_20m_0.53B` | 20m | 0.527B | 20 | 100–2000 |
| `model_20m_1.6B` | 20m | 1.58B | 50 | 100–5000 |
| `model_20m_2.6B` | 20m | 2.64B | 49 | 100–4900 |
| `model_20m_5.3B` | 20m | 5.27B | 50 | 100–5000 |
| `model_20m_11B` | 20m | 10.5B | 50 | 100–5000 |
| `model_20m_16B` | 20m | 15.8B | 30 | 100–3000 |
| `model_20m_21B` | 20m | 21.1B | 50 | 100–5000 |
| `model_20m_32B` | 20m | 31.6B | 50 | 100–5000 |
| `model_20m_40B` | 20m | 39.6B | 49 | 100–4900 |
| `model_20m_53B` | 20m | 52.7B | 32 | 100–5000 |
| `model_50m_0.23B` | 50m | 0.229B | 20 | 100–2000 |
| `model_50m_0.69B` | 50m | 0.687B | 12 | 100–1200 |
| `model_50m_1.1B` | 50m | 1.15B | 10 | 100–1000 |
| `model_50m_2.3B` | 50m | 2.29B | 20 | 100–2000 |
| `model_50m_4.6B` | 50m | 4.58B | 20 | 100–2000 |
| `model_50m_9.2B` | 50m | 9.16B | 20 | 100–2000 |
| `model_50m_17B` | 50m | 17.2B | 20 | 100–2000 |
| `model_50m_23B` | 50m | 22.9B | 10 | 100–1000 |
| `model_50m_41B` | 50m | 41.2B | 18 | 100–2000 |
| `model_680m_0.32B` | 680m | 0.319B | 5 | 100–500 |
| `model_680m_1.6B` | 680m | 1.6B | 1 | 2000 |
| `model_680m_6.4B` | 680m | 6.4B | 10 | 100–1000 |
| `model_680m_16B` | 680m | 16B | 4 | 1200–1900 |
## Not included
These models exist in the study but had no usable weights available at upload
time:
- `model_200m_0.53B`
- `model_200m_11B`
- `model_200m_21B`
- `model_200m_40B`
- `model_200m_53B`
- `model_680m_0.64B`
- `model_680m_1.2B`
- `model_680m_12B`
- `model_680m_3.2B`
- `model_680m_32B`
- 2 32m models absent from the pretraining-token tables
|