Chess GPT: board model (30M and 16M)
Transformers trained from scratch in PyTorch on 3.0M human games from Lichess to predict the move a human would play. They are never told the rules of chess and do no search: one forward pass gives a probability for each of the 1,968 possible moves.
Play against it: Chess GPT Space
| Model | Params | Human move-match top-1 | Top-5 | Probability on illegal moves | ≈ Elo vs Stockfish 18 (no search) |
|---|---|---|---|---|---|
| 30M | 31.46M | 50.8% | 87.7% | 5.1% | 1558 (95% CI 1528-1589) |
| 16M | 17.49M | 49.9% | 86.9% | 5.9% | 1492 (95% CI 1462-1522) |
How it works
Each position is turned into 81 tokens, all from the side to move's point of view. When Black is to move the board is mirrored, so the model always plays "upwards" with the uppercase pieces; its answer is mirrored back.
| Positions | Tokens | Meaning |
|---|---|---|
| 0 | <cls> |
the move is read from this position (like BERT's [CLS]) |
| 1-4 | <K>/<noK> <Q>/<noQ> <k>/<nok> <q>/<noq> |
castling rights (uppercase = mine) |
| 5 | <ep_none>, <ep_a> ... <ep_h> |
en passant file |
| 6 | <me_wins> / <me_loses> / <draw> |
game result, for the side to move |
| 7-8 | <elo_800> ... <elo_2400> |
my rating bucket, then the opponent's (200 wide) |
| 9-72 | . PNBRQK pnbrqk |
the 64 squares, a8 to h1 |
| 73-80 | UCI moves, <no_move> |
the last 8 moves, oldest first |
- Input vocab: 2,012 tokens (44 board/state tokens +
<no_move>+ 1,968 moves for the history). - Output: a softmax over all 1,968 UCI moves that can exist in chess.
- Encoder: pre-LayerNorm transformer blocks, no causal mask (every square attends to every square),
learned positional embeddings, fused
scaled_dot_product_attention. - Conditioning: the result and rating tokens are known in training (from the game) and chosen by the user at play time, so the model imitates "a game this side won, between players of these ratings".
| 30M | 16M | |
|---|---|---|
n_emb / heads / layers |
512 / 8 / 9 | 396 / 6 / 8 |
| Head size | 64 | 66 |
| Feed-forward | 512 -> 2048 -> 512 | 396 -> 1584 -> 396 |
| Parameters (incl. legal head) | 31,463,264 | 17,487,732 |
Training data
- Source: Lichess open database, May 2016 (
lichess_db_standard_rated_2016-05.pgn, 6,225,957 rated games). Lichess data is released under CC0. - Filters: no bullet games (Lichess's definition: base + 40 × increment < 180 s), only games that ended normally (no time forfeits or abandoned games, so the result token reflects the board), no very short games. 3,001,840 games kept, 224.5M moves.
- Split: 90% / 10% by game, in time order.
- Training positions: made on the fly by replaying games and taking random plies, so all ~189M training positions can be used without storing them.
- Validation set: the same 10 fixed plies from each validation game, 3,001,640 positions; the numbers here use a fixed 200,000-position sample of it.
Training
| 30M | 16M | |
|---|---|---|
| Optimizer | AdamW, betas (0.9, 0.95) | same |
| Batch | 256 positions | 256 positions |
| Learning rate | warmup 2k steps, constant 3e-4, decay to 3e-5 | warmup 2k, constant 4e-4, decay to 4e-5 |
| Steps | 280,000 (decay from 245,000) | 150,000 (decay from 130,000) |
| Positions seen | 71.7M | 38.4M |
| Precision | bf16 autocast, fp32 weights | same |
| Dropout | 0 (no position is seen twice) | 0 |
Results
Held-out positions (200,000 validation positions)
| Model | Val loss | Top-1 | Top-5 | Probability on illegal moves |
|---|---|---|---|---|
| 30M, end of constant LR (step 245k) | 1.828 | 47.6% | 85.0% | 7.0% |
| 30M, after decay (step 280k) | 1.635 | 50.8% | 87.7% | 5.1% |
| 16M, end of constant LR (step 130k) | 1.889 | 46.1% | 83.8% | 7.1% |
| 16M, after decay (step 150k) | 1.689 | 49.9% | 86.9% | 5.9% |
Top-1 = the model's most likely move is the move the human played. Top-5 = the human's move is among its top 5.
By stage of the game
How well the model does in the opening, the middle and the end of a game. 15,000 test positions, grouped by move number; plus two special groups: positions where the player is in check, and endgames (10 or fewer pieces left). "Illegal mass" = how much of its probability the model wasted on moves that aren't allowed.
| Group | Positions | 30M top-1 | 30M illegal mass | 16M top-1 | 16M illegal mass |
|---|---|---|---|---|---|
| all | 15,000 | 51.3% | 5.2% | 50.1% | 6.0% |
| moves 1-10 | 4,945 | 50.7% | 0.8% | 50.3% | 0.9% |
| moves 11-20 | 4,744 | 49.7% | 4.5% | 48.0% | 5.3% |
| moves 21-30 | 2,906 | 51.7% | 8.7% | 50.8% | 10.3% |
| moves 31+ | 2,405 | 55.1% | 11.2% | 53.0% | 12.7% |
| in check | 1,030 | 73.6% | 3.1% | 71.0% | 4.2% |
| endgame (≤ 10 pieces) | 776 | 59.8% | 10.2% | 56.6% | 11.0% |
The model's single most likely move is illegal in only 0.05% of these positions (both sizes).
Against Stockfish
The model plays its most likely legal move with no search, against Stockfish 18 at three fixed strength levels (0.1 s per move), 200 games per level. One Elo is fitted to all 600 games.
| Stockfish UCI_Elo | 30M W / D / L | 30M score | 16M W / D / L | 16M score |
|---|---|---|---|---|
| 1320 | 142 / 17 / 41 | 75.2% | 119 / 41 / 40 | 69.8% |
| 1500 | 96 / 31 / 73 | 55.8% | 71 / 32 / 97 | 43.5% |
| 1700 | 54 / 43 / 103 | 37.8% | 45 / 37 / 118 | 31.8% |
| Fitted Elo | 1558 (1528-1589) | 1492 (1462-1522) |
These are ratings on Stockfish's UCI_Elo scale, which is calibrated against engines at longer time controls;
they are not Lichess or FIDE ratings. The interval covers game-to-game noise only.
How it got here
Five model families were trained on the same data before this one; this is the last row.
| Version | Input | Top-1 | ≈ Elo vs Stockfish |
|---|---|---|---|
| Move tokens, 15.8M | the whole game as UCI move tokens (GPT, causal) | 40.5%* | - |
| Board, 16M | board from White's side, 73 tokens, 10 fixed positions per game | 36.7% | 1060 |
| Board from side to move + 8 history moves, 16M | 81 tokens, all positions on the fly | 49.0% | 1490 |
| + legal-move head, 16M | same | 49.9% | 1492 |
| + legal-move head, 30M | same | 50.8% | 1558 |
* measured on the 15,000-position subset; the others on the 200,000 positions.
Limitations
- No search. It plays what a human would likely play, one move at a time, so it misses tactics and walks into checkmates (most of its losses against Stockfish end in mate).
- Weaker after move 30. Late in the game, and especially in endgames, its play gets noticeably worse; king moves are the weakest spot (it sometimes wants to step onto attacked squares). Play with only legal moves allowed (the Space does this by default).
- Short memory. It sees the board and the last 8 moves only, so it can't plan around threefold repetition.
- Human-like, not best: it imitates 2016 Lichess players, mistakes included.
Files
| File | Contents |
|---|---|
model_30M.safetensors |
30M weights, fp32 (no optimizer state) |
config_30M.json |
hyperparameters, training step, input vocab (2,012 tokens) and output moves (1,968) |
model_16M.safetensors |
16M weights |
config_16M.json |
same for the 16M |
Usage
The network and the 81-token encoding are in the Space's
model.py
(needs torch, chess, safetensors, huggingface_hub):
import shutil
from huggingface_hub import hf_hub_download
shutil.copy(hf_hub_download("adarsh-007/chess-gpt", "model.py", repo_type="space"), "model.py")
import chess
from model import ChessModel # model.py from the Space
model = ChessModel("30M") # downloads model_30M.safetensors + config_30M.json from this repo
board = chess.Board()
board.push_san("e4")
# Ask for: Black wins, White rated 1600, Black rated 2000 (tokens are written from White's side)
probs = model.move_probs(board, "<black_wins>", "<elo_1600>", "<elo_2000>") # (1968,) over real moves
best = max(board.legal_moves, key=lambda m: probs[model.move_to_id[m.uci()]])
print(board.san(best))
License
MIT for the weights and code. Training data: Lichess open database (CC0).