chess-ooooooooo-1

A chess transformer model trained for the LLM Course Chess Challenge.

Model Architecture

This model uses a GPT-style transformer architecture optimized for chess move prediction:

• Parameters: 948,352 (0.95M)
• Vocabulary size: 85
• Embedding dimension: 128
• Number of layers: 6
• Attention heads: 4
• Feed-forward dimension: 320
• Context length: 256
• Dropout: 0.101

Training

The model was trained on a subset of the Lichess 2025 dataset, focusing on learning valid chess move sequences. The architecture was carefully tuned to stay within the 1M parameter constraint while maintaining reasonable performance.

Usage

from transformers import AutoModelForCausalLM
from src.tokenizer import ChessTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "LLM-course/chess-ooooooooo-1",
    trust_remote_code=True
)
tokenizer = ChessTokenizer.from_pretrained(
    "LLM-course/chess-ooooooooo-1",
    trust_remote_code=True
)

# Generate moves
input_text = "[BOS] WPe2e4"
input_ids = tokenizer.encode(input_text)
outputs = model.generate(input_ids, max_length=50)
predicted_moves = tokenizer.decode(outputs[0])

Submission

Submitted by etienneLefranc for the LLM Course Chess Challenge. Version 1 of chess-ooooooooo.