fsmiamoto/chess-lm

ChessLM

ChessLM is a 5.38M-parameter decoder-only Transformer trained to predict the next UCI chess move from the moves that came before it. It is an educational language-model experiment: the model receives no explicit board representation and uses no search or evaluation function.

Source code: github.com/fsmiamoto/chess-lm

Training data

The model was trained on the December 2016 Lichess standard rated archive (lichess_db_standard_rated_2016-12.pgn.zst).

Games were accepted when they:

• were rated, standard chess games with normal termination;
• had a recognized result and a valid, unique Lichess game ID;
• had both players rated at least 1800;
• had an estimated duration of at least 8 minutes (base + 40 × increment);
• contained 4–253 plies.

The first 250,000 accepted games were split deterministically by hashed game ID:

Split	Games
Train	200,124
Validation	25,144
Test	24,732

Split fingerprint: 6fef3dcb46a34758aadc7f19cd1815f435e7e431a5abc281a0545399be3b0334

Each game became one causal token sequence:

<bos> e2e4 e7e5 ... <white_win> <eos>

The result token is <white_win>, <black_win>, or <draw>. One complete UCI move is one token. Padding is excluded from the cross-entropy loss.

Model

Parameter	Value
Trainable parameters	5,379,584
Vocabulary size	4,214
Context length	256 tokens
Transformer layers	4
Attention heads	4
Model dimension	256
Feed-forward dimension	1,024
Dropout	0.1

Training configuration

Parameter	Value
Epochs	8
Batch size	32
Optimizer	AdamW
Peak learning rate	3e-4
Schedule	Linear warmup and decay
Warmup updates	1,000
Total updates	50,032
Weight decay	0.01
Maximum gradient norm	1.0
Seed	42
Device	Apple M4, 16 GB unified memory, MPS
Runtime	Python 3.12.11, PyTorch 2.13.0

Validation loss improved from 3.1815 after the first epoch to 2.4258 nats/token after the eighth (perplexity 11.31). The uploaded artifact is that best-validation checkpoint.

Training source revision: cdcee31

Evaluation

Moves were selected deterministically as the highest-logit legal UCI move. In 44 games against Stockfish 18 at Skill Level 0 with a 1 ms move limit, alternating colors, ChessLM scored 1 win, 11 draws, and 32 losses. Its win was as Black by checkmate in 26 plies.

This is a small, intentionally weak-engine evaluation—not a reliable Elo estimate.

Using the weights

best.weights.pt is a custom, self-describing ChessLM artifact rather than a Transformers checkpoint. Clone the source repository so the matching model code and vocabulary are available:

from chess_lm.checkpoint import load_portable_weights
from chess_lm.vocabulary import Vocabulary

model = load_portable_weights("best.weights.pt", Vocabulary(), device="cpu")

Artifact details:

• format: chess_lm.weights, version 1;
• selected checkpoint: update 50,032, epoch index 7;
• source archive SHA-256: 285d09bcb7f47af2d58594ecee7da8c18f9e800fd7df2e0158b4fb5429f9a904;
• vocabulary SHA-256: 5cfd8ba2d809eef77a7d4c189408ab33f477021e032fe1e6cda9de0f08eaff2d;
• file SHA-256: d1dabbae3576f820ed5185d3859349ffa82b364eeaa3cf6ecb43b04d991dc917.

Limitations

ChessLM imitates moves in its training data; it does not reason over an explicit board or search future positions. Legal-move filtering is required during play. Its data comes from one month and one rating threshold, its context is limited to 256 tokens, and the Stockfish result shows that it should not be treated as a competitive chess engine.