Create README.md

README.md

@@ -0,0 +1,112 @@
+---
+language: en
+tags:
+- chess
+- transformer
+- pytorch
+- move-prediction
+- from-to-heads
+- recurrent-transformer
+datasets:
+- angeluriot/chess_games
+metrics:
+- cross-entropy
+- move-accuracy
+---
+
+# Recurrent Transformer for Chess Move Prediction (v1)
+
+Recurrent transformer that predicts the best chess move from a board state (FEN). Trained from scratch; uses From/To square prediction heads and a shared transformer block applied 8 times with iteration embeddings (CORnet-s / Universal Transformer style). Output is always a legal move (zero fallbacks).
+
+**Part of:** INFOMTALC 2025/26 (Utrecht University, MSc Applied Data Science) — chess tournament assignment.
+
+---
+
+## Model description
+
+- **Architecture:** Encoder-only transformer with recurrent weight sharing. Board is encoded as 70 tokens (turn, castling×4, 64 squares, en passant) using a spatial tokenizer (14 piece IDs). Separate embeddings per token type; one shared block (multi-head self-attention + feed-forward with GELU) applied 8 times with learned iteration embeddings. Two linear heads score each of the 64 squares as source and destination; each legal move is scored as `from_logits[src] + to_logits[dst]` and the best is returned.
+- **Inspiration:** From/To heads from [sgrvinod/chess-transformers](https://github.com/sgrvinod/chess-transformers); recurrent shared block from CORnet-s (Kubilius et al., 2019) and BLT networks (Spoerer et al., 2017).
+- **Parameters:** ~3.2M (d_model=512, 8 heads, FFN=2048, 8 iterations).
+- **Input:** FEN string.
+- **Output:** Single UCI move string (e.g. `e2e4`). No sampling; deterministic given the position.
+
+---
+
+## Intended use
+
+- **Intended:** Chess move prediction for the INFOMTALC tournament and similar setups (player that receives FEN and returns UCI move). Runs on CPU or GPU; fits on free-tier Colab (T4).
+- **Not intended:** General-purpose chess engine, opening book, or strength comparable to Stockfish. This is a small transformer trained on engine labels, not a full engine.
+
+---
+
+## Training data
+
+- **Source:** [angeluriot/chess_games](https://huggingface.co/datasets/angeluriot/chess_games) (games filtered to ELO ≥ 1500).
+- **Labels:** Best move per position from local Stockfish (depth 10).
+- **Size:** 750K (FEN, UCI) pairs in JSONL format.
+
+---
+
+## Training procedure
+
+- **Loss:** Cross-entropy on from-square and to-square predictions separately.
+- **Optimizer:** AdamW with Vaswani-style learning rate schedule (warmup 4000 steps).
+- **Batch size:** 512.
+- **Epochs:** 16 with early stopping (patience 2).
+- **Hardware:** Trained locally (5070Ti). Inference runs on free-tier Colab T4.
+
+---
+
+## How to use
+
+Requires the [chess_exam](https://github.com/bylinina/chess_exam) package (for the `Player` base class and `Game`). Install it, then use the model via the tournament player class from the assignment repo:
+
+```python
+# Install tournament framework
+# git clone https://github.com/bylinina/chess_exam.git && cd chess_exam && pip install -e .
+
+from chess_tournament import Game, RandomPlayer
+from player import TransformerPlayer  # from the assignment repo that contains model.py + player.py
+
+tp = TransformerPlayer("RecurrentTransformer")  # downloads this model from HF on first use
+rp = RandomPlayer("Random")
+game = Game(tp, rp, max_half_moves=200)
+outcome, scores, fallbacks = game.play()
+print(outcome, fallbacks)
+```
+
+**Loading only the PyTorch state dict (no player):**
+
+```python
+import json
+import torch
+from huggingface_hub import hf_hub_download
+from model import RecurrentTransformer  # need model.py from the assignment repo
+
+config_path = hf_hub_download("Izzent/recurrent-transformer-chess", "config.json")
+weights_path = hf_hub_download("Izzent/recurrent-transformer-chess", "model.pt")
+
+with open(config_path) as f:
+    config = json.load(f)
+
+model = RecurrentTransformer.from_config(config)
+state = torch.load(weights_path, map_location="cpu", weights_only=True)
+model.load_state_dict(state)
+model.eval()
+
+# Forward pass expects a batch dict: board (B,64), turn (B,1), castling (B,4), ep (B,1)
+# Use BoardTokenizer.encode(fen) to get these from a FEN string.
+```
+
+---
+
+## Files
+
+- `config.json`: Model config (d_model, nhead, d_ff, num_iterations, dropout).
+- `model.pt`: PyTorch state dict (weights only).
+
+---
+
+## License
+
+All rights reserved.