chess-moves-2400plus

The move-level twin of chess-pgn-2400plus: the same 51M-parameter transformer, data and training recipe, except that it reads and writes one token per move instead of one per character. It is trained only on Lichess games where both players were rated 2400+, and is never shown a board or the rules.

Tokens

The whole UCI move space, not just the moves in the data: 1,792 from-to pairs a queen or knight could make (castling is e1g1 etc.), 176 promotions, plus ; (start of game) and the results 1-0, 0-1, 1/2-1/2: 1,972 tokens (move_tokens.py). A game is ; e2e4 e7e5 g1f3 ... 1-0. Nothing is masked: the model predicts over all 1,972 tokens and learns legality from the games.

Data

Exactly the character model's games, in the same order: 10,000,000 training games and the same 5,000 held out (Lichess, March 2023 to June 2025, both players 2400+, base time >= 180 s, draws kept), converted move for move from its SAN (88.4 moves per game on average).

Model

Decoder-only transformer: 16 layers, d=512, 8 heads, ReLU MLPs, learned positions, context 184 moves (52.6M parameters; the character model's embedding and output layer cover 33 tokens, these 1,972).

Training

As the character model: next-token cross-entropy on windows that each start at a game's beginning, 128 windows per step. The window, 184 moves, holds as many games as the character model's 1,024 characters, so a step sees the same ~270 games. AdamW (0.9/0.95, weight decay 0.1), lr 2e-4 after 2,000 warmup steps, cosine to 2e-5 over 200,000 steps, bf16, one H100; then the character model's three fine-tuning stages, each from the previous stage's best weights with a fresh optimizer, ended by early stopping (peak lr 5e-5, 2e-5, then 1e-5 after a 20,000-step warmup).

stage best after (steps) held-out loss (nats per token)
base 199,500 1.3801
-ft1 500 1.3767
-ft2 1,000 1.3749
-ft3 12,500 1.3682

This checkpoint: moves-2400+-ft3, 213,500 optimizer steps on its path.

Compared with the character model

The same 4,875 held-out games (417,045 moves) scored whole by both models (compare_twin.py), with this model at moves-2400+-ft3 (this checkpoint). For the character model, a move's cost is the sum over its SAN characters and the space that ends it.

chess-pgn-2400plus (characters) this model (moves)
bits per game 171.7 173.0
surprise of the played move (nats) 1.3587 1.3728
result (nats) 2.418 2.467
top move = move played 54.9%

Usage

import json, chess, sys
from huggingface_hub import snapshot_download
from safetensors.torch import load_file

path = snapshot_download("Arisp/chess-moves-2400plus")
sys.path.insert(0, path)
from pgn_model import PGNTransformer
from move_tokens import generate_moves

cfg = json.load(open(f"{path}/config.json"))["architecture"]
model = PGNTransformer(**cfg)
model.load_state_dict(load_file(f"{path}/model.safetensors"))
model.eval()

board = chess.Board()
board.push_san("e4")
move = generate_moves(model, [board], temperature=0.0, dev="cpu")[0]   # None if 5 attempts were illegal
print(board.san(move))

Limitations

It knows the position only from the move history and can predict an illegal move (it is resampled, up to 5 tries); games longer than 184 moves (about 2%) don't fit its context. It plays like a strong human of its band, blunders included; it is not an engine.

Downloads last month
37
Safetensors
Model size
52.6M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support